Health monitoring system

ABSTRACT

Systems and methods provide automatic messaging to a client on behalf of a healthcare treatment professional by setting up one or more computer implemented agents, each specializing in a disease state, with rules to respond to a client condition; during run-time, receiving a communication from the client and in response selecting one or more computer implemented agents to respond to the communication; and automatically generating a response to be rendered on a client mobile device to encourage healthy behavior.

This application is a continuation of U.S. application Ser. No.13/707,682, filed Dec. 7, 2012, the content of which is incorporated byreference.

BACKGROUND

This invention relates generally to methods and apparatus forcommunicating health instructions to users. Healthcare costs around theworld have been rising. One reason is that, obesity is common, seriousand costly. A Duke University study suggests that by 2030, about 42% ofAmericans will be obese, which is up from 36% in 2012 and will costabout $550 billion dollars. Even small reductions in obesity prevalence“could result in substantial savings,” wrote the authors.Obesity-related conditions increase the odds of heart disease, stroke,type 2 diabetes and certain types of cancer, some of the leading causesof preventable death. In 2008, medical costs associated with obesitywere estimated at $147 billion; the medical costs for people who areobese were $1,429 higher than those of normal weight.

Obesity affects some groups more than others. Non-Hispanic blacks havethe highest age-adjusted rates of obesity (49.5%) compared with MexicanAmericans (40.4%), all Hispanics (39.1%) and non-Hispanic whites(34.3%). Among non-Hispanic black and Mexican-American men, those withhigher incomes are more likely to be obese than those with low income.Higher income women are less likely to be obese than low-income women.There is no significant relationship between obesity and education amongmen. Among women, however, there is a trend—those with college degreesare less likely to be obese compared with less educated women. Thus,education appears to be key. Between 1988-1994 and 2007-2008 theprevalence of obesity increased in adults at all income and educationlevels.

A government solution has been suggested. For example, a ban on the useof trans fats in NY restaurants has sharply reduced the consumption ofthese unhealthy fats among fast-food customers. However, the governmentand regulation may not be the best way to solve the problem.

To treat obesity in a cost effective manner, coordination is neededamong different service providers such as dieticians, doctors, andexercise coaches. However, planning information, alerts and remindersmay be haphazardly and intermittently distributed to doctors,clinicians, or their staff with existing healthcare appointment andscheduling systems and do not support multi-vendor calendaring systemthat shares information among the different providers. This occurs inother treatments as well. For example, in the case of a patientscheduled for radiation therapy, an existing system may be aware of anecessary number of appointments and treatment orders, but these numberstypically are not compatible with the (e.g., one or more) treatmentplans involved. Consequently, a clinician needs to work out a referralconnection manually for each appointment. The existing systems alsorequire manual coordination of appointments with treatment plan goalsand treatment plan results which occupies a significant amount ofclinician time in gathering, collating and analyzing information.

One way to monitor the impact of obesity is to monitor blood pressure.As discussed in U.S. Pat. No. 6,514,211, three well known techniqueshave been used to non-invasively monitor a subject's arterial bloodpressure waveform: auscultation, oscillometry, and tonometry. Theauscultation and oscillometry techniques use a standard inflatable armcuff that occludes the subject's brachial artery. The auscultatorytechnique determines the subject's systolic and diastolic pressures bymonitoring certain Korotkoff sounds that occur as the cuff is slowlydeflated. The oscillometric technique, on the other hand, determinesthese pressures, as well as the subject's mean pressure, by measuringactual pressure changes that occur in the cuff as the cuff is deflated.Both techniques determine pressure values only intermittently, becauseof the need to alternately inflate and deflate the cuff, and they cannotreplicate the subject's actual blood pressure waveform. Occlusive cuffinstruments of the kind described briefly above generally have beeneffective in sensing long-term trends in a subject's blood pressure, butthey have been ineffective in sensing short-term blood pressurevariations.

The '211 patent discloses blood pressure measurement by determining themean arterial blood pressure (MAP) of a subject during tonometricconditions. The apparatus has one or more pressure and ultrasoundtransducers placed over the radial artery of a human subject's wrist,the latter transmitting and receiving acoustic energy so as to permitthe measurement of blood velocity during periods of variable compressionof the artery. During compression, the ultrasound velocity waveforms arerecorded and processed using time-frequency analysis. The time at whichthe mean time-frequency distribution is maximal corresponds to the timeat which the transmural pressure equals zero, and the mean pressure readby the transducer equals the mean pressure within the artery. In anotheraspect of the invention, the ultrasound transducer is used to positionthe transducer over the artery such that the accuracy of the measurementis maximized.

SUMMARY

Systems and methods provide automatic messaging to a client on behalf ofa healthcare treatment professional by setting up one or more computerimplemented agents, each specializing in a disease state, with rules torespond to a client condition; during run-time, receiving acommunication from the client and in response selecting one or morecomputer implemented agents to respond to the communication; andautomatically generating a response to be rendered on a client mobiledevice to encourage healthy behavior.

Advantages of the preferred embodiment may include one or more of thefollowing. The system turns obesity and overweight into preventablephenomena, and the myriad health problems associated with them. Thesystem not saves money and enhances lives by continued efforts atgetting the weight off. The system brings awareness of the serioushealth issues—physical and mental—that are linked to being overweight orobese, along with the tried and true lifestyle changes like diet andexercise, as ways for battling the obesity epidemic.

In another aspect, a heart monitoring system for a person includes oneor more wireless nodes; and a wearable appliance in communication withthe one or more wireless nodes, the appliance continuously monitoringpatient vital signs or other data such as cardiac abnormalities.Embodiments can monitor heart rate, heart rate variability, respiratoryrate, fluid status, posture and activity.

In a further aspect, a monitoring system for a person includes one ormore wireless nodes; and a wristwatch having a wireless transceiveradapted to communicate with the one or more wireless nodes; and anaccelerometer to detect a dangerous condition and to generate a warningwhen the dangerous condition is detected.

In yet another aspect, a heart monitoring system for a person includesone or more wireless nodes forming a wireless network and a wearableappliance having a sound transducer coupled to the wireless transceiver;and a heart disease recognizer coupled to the sound transducer todetermine cardiovascular health and to transmit heart sound over thewireless network to a remote listener if the recognizer identifies acardiovascular problem. The heart sound being transmitted may becompressed to save transmission bandwidth.

In another aspect, a monitoring system for a person includes one or morewireless nodes forming a wireless network; and a wearable appliancehaving a wireless transceiver adapted to communicate with the one ormore wireless nodes; and a heartbeat detector coupled to the wirelesstransceiver. Embodiments may also include an accelerometer to detect adangerous condition such as a falling condition and to generate awarning when the dangerous condition is detected.

Implementations of the above aspect may include one or more of thefollowing. The wristwatch determines position based on triangulation.The wristwatch determines position based on RF signal strength and RFsignal angle. A switch detects a confirmatory signal from the person.The confirmatory signal includes a head movement, a hand movement, or amouth movement. The confirmatory signal includes the person's voice. Aprocessor in the system executes computer readable code to transmit ahelp request to a remote computer. The code can encrypt or scramble datafor privacy. The processor can execute voice over IP (VOIP) code toallow a user and a remote person to audibly communicate with each other.The voice communication system can include Zigbee VOIP or Bluetooth VOIPor 802.XX VOIP. The remote person can be a doctor, a nurse, a medicalassistant, or a caregiver. The system includes code to store and analyzepatient information. The patient information includes medicine takinghabits, eating and drinking habits, sleeping habits, or excise habits. Apatient interface is provided on a user computer for accessinginformation and the patient interface includes in one implementation atouch screen; voice-activated text reading; and one touch telephonedialing. The processor can execute code to store and analyze informationrelating to the person's ambulation. A global positioning system (GPS)receiver can be used to detect movement and where the person falls. Thesystem can include code to map the person's location onto an area forviewing. The system can include one or more cameras positioned tocapture three dimensional (3D) video of the patient; and a servercoupled to the one or more cameras, the server executing code to detecta dangerous condition for the patient based on the 3D video and allow aremote third party to view images of the patient when the dangerouscondition is detected.

In another aspect, a monitoring system for a person includes one or morewireless bases; and a cellular telephone having a wireless transceiveradapted to communicate with the one or more wireless bases; and anaccelerometer to detect a dangerous condition and to generate a warningwhen the dangerous condition is detected.

In yet another aspect, a monitoring system includes one or more camerasto determine a three dimensional (3D) model of a person; means to detecta dangerous condition based on the 3D model; and means to generate awarning when the dangerous condition is detected.

In another aspect, a method to detect a dangerous condition for aninfant includes placing a pad with one or more sensors in the infant'sdiaper; collecting infant vital parameters; processing the vitalparameter to detect SIDS onset; and generating a warning.

Advantages of these embodiments may include one or more of thefollowing. The system for non-invasively and continually monitors asubject's arterial blood pressure, with reduced susceptibility to noiseand subject movement, and relative insensitivity to placement of theapparatus on the subject. The system does not need frequentrecalibration of the system while in use on the subject.

In particular, it allows patients to conduct a low-cost, comprehensive,real-time monitoring of their blood pressure. Using the web servicessoftware interface, the invention then avails this information tohospitals, home-health care organizations, insurance companies,pharmaceutical agencies conducting clinical trials and otherorganizations. Information can be viewed using an Internet-basedwebsite, a personal computer, or simply by viewing a display on themonitor. Data measured several times each day provide a relativelycomprehensive data set compared to that measured during medicalappointments separated by several weeks or even months. This allows boththe patient and medical professional to observe trends in the data, suchas a gradual increase or decrease in blood pressure, which may indicatea medical condition. The invention also minimizes effects of white coatsyndrome since the monitor automatically makes measurements withbasically no discomfort; measurements are made at the patient's home orwork, rather than in a medical office.

The wearable appliance is small, easily worn by the patient duringperiods of exercise or day-to-day activities, and non-invasivelymeasures blood pressure can be done in a matter of seconds withoutaffecting the patient. An on-board or remote processor can analyze thetime-dependent measurements to generate statistics on a patient's bloodpressure (e.g., average pressures, standard deviation, beat-to-beatpressure variations) that are not available with conventional devicesthat only measure systolic and diastolic blood pressure at isolatedtimes.

Other advantages of the invention may include one or more of thefollowing. The wearable appliance provides an in-depth, cost-effectivemechanism to evaluate a patient's cardiac condition. Certain cardiacconditions can be controlled, and in some cases predicted, before theyactually occur. Moreover, data from the patient can be collected andanalyzed while the patient participates in their normal, day-to-dayactivities.

In cases where the device has fall detection in addition to bloodpressure measurement, other advantages of the invention may include oneor more of the following. The system provides timely assistance andenables elderly and disabled individuals to live relatively independentlives. The system monitors physical activity patterns, detects theoccurrence of falls, and recognizes body motion patterns leading tofalls. Continuous monitoring of patients is done in an accurate,convenient, unobtrusive, private and socially acceptable manner since acomputer monitors the images and human involvement is allowed only underpre-designated events. The patient's privacy is preserved since humanaccess to videos of the patient is restricted: the system only allowshuman viewing under emergency or other highly controlled conditionsdesignated in advance by the user. When the patient is healthy, peoplecannot view the patient's video without the patient's consent. Only whenthe patient's safety is threatened would the system provide patientinformation to authorized medical providers to assist the patient. Whenan emergency occurs, images of the patient and related medical data canbe compiled and sent to paramedics or hospital for proper preparationfor pick up and check into emergency room.

The system allows certain designated people such as a family member, afriend, or a neighbor to informally check on the well-being of thepatient. The system is also effective in containing the spiraling costof healthcare and outpatient care as a treatment modality by providingremote diagnostic capability so that a remote healthcare provider (suchas a doctor, nurse, therapist or caregiver) can visually communicatewith the patient in performing remote diagnosis. The system allowsskilled doctors, nurses, physical therapists, and other scarce resourcesto assist patients in a highly efficient manner since they can do themajority of their functions remotely.

Additionally, a sudden change of activity (or inactivity) can indicate aproblem. The remote healthcare provider may receive alerts over theInternet or urgent notifications over the phone in case of such suddenaccident indicating changes. Reports of health/activity indicators andthe overall well being of the individual can be compiled for the remotehealthcare provider. Feedback reports can be sent to monitored subjects,their designated informal caregiver and their remote healthcareprovider. Feedback to the individual can encourage the individual toremain active. The content of the report may be tailored to the targetrecipient's needs, and can present the information in a formatunderstandable by an elder person unfamiliar with computers, via anappealing patient interface. The remote healthcare provider will haveaccess to the health and well- being status of their patients withoutbeing intrusive, having to call or visit to get such informationinterrogatively. Additionally, remote healthcare provider can receive areport on the health of the monitored subjects that will help themevaluate these individuals better during the short routine check upvisits. For example, the system can perform patient behavior analysissuch as eating/drinking/smoke habits and medication compliance, amongothers.

Yet other advantages of the system may include one or more of thefollowing. The patient's home equipment is simple to use and modular toallow for the accommodation of the monitoring device to the specificneeds of each patient. Moreover, the system is simple to install.Regular monitoring of the basic wellness parameters provides significantbenefits in helping to capture adverse events sooner, reduce hospitaladmissions, and improve the effectiveness of medications, hence,lowering patient care costs and improving the overall quality of care.Suitable users for such systems are disease management companies, healthinsurance companies, self-insured employers, medical devicemanufacturers and pharmaceutical firms.

The system reduces costs by automating data collection and compliancemonitoring, and hence reduce the cost of nurses for hospital and nursinghome applications. At-home vital signs monitoring enables reducedhospital admissions and lower emergency room visits of chronic patients.Operators in the call centers or emergency response units get highquality information to identify patients that need urgent care so thatthey can be treated quickly, safely, and cost effectively. The Web basedtools allow easy access to patient information for authorized partiessuch as family members, neighbors, physicians, nurses, pharmacists,caregivers, and other affiliated parties to improved Quality of Care forthe patient.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary system for monitoring a person.

FIG. 2A illustrates a process for determining three dimensional (3D)detection while FIG. 2B shows an exemplary calibration sheet.

FIG. 3 illustrates a process for detecting patient exercise andactivity.

FIG. 4 illustrates a process for detecting patient movements forexercise tracking.

FIG. 5 illustrates an exemplary process for determining and gettingassistance for a patient or user.

FIG. 6A shows an exemplary wrist-watch based assistance device.

FIG. 6B shows an exemplary mesh network working with the wearableappliance of FIG. 6A.

FIG. 7 shows an exemplary mesh network in communication with thewrist-watch device of FIG. 6.

FIGS. 8-14 show various exemplary wearable appliances to monitor apatient.

FIGS. 15A-15B show exemplary systems for performing patient monitoring,FIG. 15C shows an exemplary interface to monitor a plurality of persons,FIG. 15D shows an exemplary dash-board that provides summary informationon the status of a plurality of persons, FIG. 15E shows an exemplarymulti-station vital parameter user interface for a professionalembodiment, FIG. 15F shows an exemplary trending pattern display.

FIGS. 16A-16B show exemplary blood pressure determination processes.

FIG. 17A shows an exemplary process for automated interactivecommunication between clinicians and patients, FIG. 17B shows oneexemplary weight loss treatment algorithm using the process of FIG. 17A,FIG. 17C shows in more details the event handler of FIG. 17A, FIG. 17Dshows exemplary agents for one treatment scenario.

FIG. 18A shows another exemplary process for monitoring a patient, FIG.18B shows an exemplary process for monitoring patient food intake, andFIG. 18C shows an exemplary exercise recommendation and monitoringprocess.

FIG. 19 shows an exemplary exercise monitoring system.

FIG. 20 shows exemplary environment for collecting sensor data todetermine a status of a user.

FIG. 21 shows different modules of portable electronic device asdescribed in the FIG. 20.

FIGS. 22 a and 22 b shows exemplary dynamic and static habits of theuser.

FIG. 23 shows an exemplary preventive health care recommendation tables.

FIG. 24 shows exemplary operations performed by the system as describedin the FIGS. 20 and 21.

FIG. 25 shows a flow chart illustrating a method for providingpreventive health care recommendations to the user.

DESCRIPTION

FIG. 1 shows an exemplary patient monitoring system. The system canoperate in a home, a nursing home, or a hospital. In this system, one ormore mesh network appliances 8 are provided to enable wirelesscommunication in the home monitoring system. Appliances 8 in the meshnetwork can include home security monitoring devices, door alarm, windowalarm, home temperature control devices, fire alarm devices, amongothers. Appliances 8 in the mesh network can be one of multiple portablephysiological transducer, such as a blood pressure monitor, heart ratemonitor, weight scale, thermometer, spirometer, single or multiple leadelectrocardiograph (ECG), a pulse oxymeter, a body fat monitor, acholesterol monitor, a signal from a medicine cabinet, a signal from adrug container, a signal from a commonly used appliance such as arefrigerator/stove/oven/washer, or a signal from an exercise machine,such as a heart rate. As will be discussed in more detail below, oneappliance is a patient monitoring device that can be worn by the patientand includes a single or bi-directional wireless communication link,generally identified by the bolt symbol in FIG. 1, for transmitting datafrom the appliances 8 to the local hub or receiving station or basestation server 20 by way of a wireless radio frequency (RF) link using aproprietary or non-proprietary protocol. For example, within a house, auser may have mesh network appliances that detect window and doorcontacts, smoke detectors and motion sensors, video cameras, key chaincontrol, temperature monitors, CO and other gas detectors, vibrationsensors, and others. A user may have flood sensors and other detectorson a boat. An individual, such as an ill or elderly grandparent, mayhave access to a panic transmitter or other alarm transmitter. Othersensors and/or detectors may also be included. The user may registerthese appliances on a central security network by entering theidentification code for each registered appliance/device and/or system.The mesh network can be Zigbee network or 802.15 network. More detailsof the mesh network is shown in FIG. 7 and discussed in more detailbelow.

A plurality of monitoring cameras 10 may be placed in variouspredetermined positions in a home of a patient 30. The cameras 10 can bewired or wireless. For example, the cameras can communicate overinfrared links or over radio links conforming to the 802X (e.g. 802.11A,802.11B, 802.11G, 802.15) standard or the Bluetooth standard to a basestation/server 20 may communicate over various communication links, suchas a direct connection, such a serial connection, USB connection,Firewire connection or may be optically based, such as infrared orwireless based, for example, home RF, IEEE standard 802.11 a/b,Bluetooth or the like. In one embodiment, appliances 8 monitor thepatient and activates the camera 10 to capture and transmit video to anauthorized third party for providing assistance should the appliance 8detects that the user needs assistance or that an emergency hadoccurred.

The base station/server 20 stores the patient's ambulation pattern andvital parameters and can be accessed by the patient's family members(sons/daughters), physicians, caretakers, nurses, hospitals, and elderlycommunity. The base station/server 20 may communicate with the remoteserver 200 by DSL, T-1 connection over a private communication networkor a public information network, such as the Internet 100, among others.

The patient 30 may wear one or more wearable patient monitoringappliances such as wrist-watches or clip on devices or electronicjewelry to monitor the patient. One wearable appliance such as awrist-watch includes sensors 40, for example devices for sensing ECG,EKG, blood pressure, sugar level, among others. In one embodiment, thesensors 40 are mounted on the patient's wrist (such as a wristwatchsensor) and other convenient anatomical locations. Exemplary sensors 40include standard medical diagnostics for detecting the body's electricalsignals emanating from muscles (EMG and EOG) and brain (EEG) andcardiovascular system (ECG). Leg sensors can include piezoelectricaccelerometers designed to give qualitative assessment of limb movement.Additionally, thoracic and abdominal bands used to measure expansion andcontraction of the thorax and abdomen respectively. A small sensor canbe mounted on the subject's finger in order to detect blood-oxygenlevels and pulse rate. Additionally, a microphone can be attached tothroat and used in sleep diagnostic recordings for detecting breathingand other noise. One or more position sensors can be used for detectingorientation of body (lying on left side, right side or back) duringsleep diagnostic recordings. Each of sensors 40 can individuallytransmit data to the server 20 using wired or wireless transmission.Alternatively, all sensors 40 can be fed through a common bus into asingle transceiver for wired or wireless transmission. The transmissioncan be done using a magnetic medium such as a floppy disk or a flashmemory card, or can be done using infrared or radio network link, amongothers. The sensor 40 can also include an indoor positioning system oralternatively a global position system (GPS) receiver that relays theposition and ambulatory patterns of the patient to the server 20 formobility tracking.

In one embodiment, the sensors 40 for monitoring vital signs areenclosed in a wrist-watch sized case supported on a wrist band. Thesensors can be attached to the back of the case. For example, in oneembodiment, Cygnus' AutoSensor (Redwood City, Calif.) is used as aglucose sensor. A low electric current pulls glucose through the skin.Glucose is accumulated in two gel collection discs in the AutoSensor.The AutoSensor measures the glucose and a reading is displayed by thewatch.

In another embodiment, EKG/ECG contact points are positioned on the backof the wrist-watch case. In yet another embodiment that providescontinuous, beat-to-beat wrist arterial pulse rate measurements, apressure sensor is housed in a casing with a ‘free-floating’ plunger asthe sensor applanates the radial artery. A strap provides a constantforce for effective applanation and ensuring the position of the sensorhousing to remain constant after any wrist movements. The change in theelectrical signals due to change in pressure is detected as a result ofthe piezoresistive nature of the sensor are then analyzed to arrive atvarious arterial pressure, systolic pressure, diastolic pressure, timeindices, and other blood pressure parameters.

The case may be of a number of variations of shape but can beconveniently made a rectangular, approaching a box-like configuration.The wrist-band can be an expansion band or a wristwatch strap ofplastic, leather or woven material. The wrist-band further contains anantenna for transmitting or receiving radio frequency signals. Thewristband and the antenna inside the band are mechanically coupled tothe top and bottom sides of the wrist-watch housing. Further, theantenna is electrically coupled to a radio frequency transmitter andreceiver for wireless communications with another computer or anotheruser. Although a wrist-band is disclosed, a number of substitutes may beused, including a belt, a ring holder, a brace, or a bracelet, amongother suitable substitutes known to one skilled in the art. The housingcontains the processor and associated peripherals to provide thehuman-machine interface. A display is located on the front section ofthe housing. A speaker, a microphone, and a plurality of push-buttonswitches and are also located on the front section of housing. Aninfrared LED transmitter and an infrared LED receiver are positioned onthe right side of housing to enable the watch to communicate withanother computer using infrared transmission.

In another embodiment, the sensors 40 are mounted on the patient'sclothing. For example, sensors can be woven into a single-piece garment(an undershirt) on a weaving machine. A plastic optical fiber can beintegrated into the structure during the fabric production processwithout any discontinuities at the armhole or the seams. Aninterconnection technology transmits information from (and to) sensorsmounted at any location on the body thus creating a flexible “bus”structure. T-Connectors—similar to “button clips” used in clothing—areattached to the fibers that serve as a data bus to carry the informationfrom the sensors (e.g., EKG sensors) on the body. The sensors will pluginto these connectors and at the other end similar T-Connectors will beused to transmit the information to monitoring equipment or personalstatus monitor. Since shapes and sizes of humans will be different,sensors can be positioned on the right locations for all patients andwithout any constraints being imposed by the clothing. Moreover, theclothing can be laundered without any damage to the sensors themselves.In addition to the fiber optic and specialty fibers that serve assensors and data bus to carry sensory information from the wearer to themonitoring devices, sensors for monitoring the respiration rate can beintegrated into the structure.

In another embodiment, instead of being mounted on the patient, thesensors can be mounted on fixed surfaces such as walls or tables, forexample. One such sensor is a motion detector. Another sensor is aproximity sensor. The fixed sensors can operate alone or in conjunctionwith the cameras 10. In one embodiment where the motion detectoroperates with the cameras 10, the motion detector can be used to triggercamera recording. Thus, as long as motion is sensed, images from thecameras 10 are not saved. However, when motion is not detected, theimages are stored and an alarm may be generated. In another embodimentwhere the motion detector operates stand alone, when no motion issensed, the system generates an alarm.

The server 20 also executes one or more software modules to analyze datafrom the patient. A module 50 monitors the patient's vital signs such asECG/EKG and generates warnings should problems occur. In this module,vital signs can be collected and communicated to the server 20 usingwired or wireless transmitters. In one embodiment, the server 20 feedsthe data to a statistical analyzer such as a neural network which hasbeen trained to flag potentially dangerous conditions. The neuralnetwork can be a back-propagation neural network, for example. In thisembodiment, the statistical analyzer is trained with training data wherecertain signals are determined to be undesirable for the patient, givenhis age, weight, and physical limitations, among others. For example,the patient's glucose level should be within a well established range,and any value outside of this range is flagged by the statisticalanalyzer as a dangerous condition. As used herein, the dangerouscondition can be specified as an event or a pattern that can causephysiological or psychological damage to the patient. Moreover,interactions between different vital signals can be accounted for sothat the statistical analyzer can take into consideration instanceswhere individually the vital signs are acceptable, but in certaincombinations, the vital signs can indicate potentially dangerousconditions. Once trained, the data received by the server 20 can beappropriately scaled and processed by the statistical analyzer. Inaddition to statistical analyzers, the server 20 can process vital signsusing rule-based inference engines, fuzzy logic, as well as conventionalif-then logic. Additionally, the server can process vital signs usingHidden Markov Models (HMMs), dynamic time warping, or template matching,among others.

Through various software modules, the system reads video sequence andgenerates a 3D anatomy file out of the sequence. The proper bone andmuscle scene structure are created for head and face. A based profilestock phase shape will be created by this scene structure. Every scenewill then be normalized to a standardized viewport.

A module 52 monitors the patient ambulatory pattern and generateswarnings should the patient's patterns indicate that the patient hasfallen or is likely to fall. 3D detection is used to monitor thepatient's ambulation. In the 3D detection process, by putting 3 or moreknown coordinate objects in a scene, camera origin, view direction andup vector can be calculated and the 3D space that each camera views canbe defined.

In one embodiment with two or more cameras, camera parameters (e.g.field of view) are preset to fixed numbers. Each pixel from each cameramaps to a cone space. The system identifies one or more 3D featurepoints (such as a birthmark or an identifiable body landmark) on thepatient. The 3D feature point can be detected by identifying the samepoint from two or more different angles. By determining the intersectionfor the two or more cones, the system determines the position of thefeature point. The above process can be extended to certain featurecurves and surfaces, e.g. straight lines, arcs; flat surfaces,cylindrical surfaces. Thus, the system can detect curves if a featurecurve is known as a straight line or arc. Additionally, the system candetect surfaces if a feature surface is known as a flat or cylindricalsurface. The further the patient is from the camera, the lower theaccuracy of the feature point determination. Also, the presence of morecameras would lead to more correlation data for increased accuracy infeature point determination. When correlated feature points, curves andsurfaces are detected, the remaining surfaces are detected by texturematching and shading changes. Predetermined constraints are appliedbased on silhouette curves from different views. A different constraintcan be applied when one part of the patient is occluded by anotherobject. Further, as the system knows what basic organic shape it isdetecting, the basic profile can be applied and adjusted in the process.

In a single camera embodiment, the 3D feature point (e.g. a birth mark)can be detected if the system can identify the same point from twoframes. The relative motion from the two frames should be small butdetectable. Other features curves and surfaces will be detectedcorrespondingly, but can be tessellated or sampled to generate morefeature points. A transformation matrix is calculated between a set offeature points from the first frame to a set of feature points from thesecond frame. When correlated feature points, curves and surfaces aredetected, the rest of the surfaces will be detected by texture matchingand shading changes.

Each camera exists in a sphere coordinate system where the sphere origin(0,0,0) is defined as the position of the camera. The system detectstheta and phi for each observed object, but not the radius or size ofthe object. The radius is approximated by detecting the size of knownobjects and scaling the size of known objects to the object whose sizeis to be determined. For example, to detect the position of a ball thatis 10 cm in radius, the system detects the ball and scales otherfeatures based on the known ball size. For human, features that areknown in advance include head size and leg length, among others. Surfacetexture can also be detected, but the light and shade information fromdifferent camera views is removed. In either single or multiple cameraembodiments, depending on frame rate and picture resolution, certainundetected areas such as holes can exist. For example, if the patientyawns, the patient's mouth can appear as a hole in an image. For 3Dmodeling purposes, the hole can be filled by blending neighborhoodsurfaces. The blended surfaces are behind the visible line.

In one embodiment shown in FIG. 2A, each camera is calibrated before 3Ddetection is done. Pseudo-code for one implementation of a cameracalibration process is as follows:

-   -   Place a calibration sheet with known dots at a known distance        (e.g. 1 meter), and perpendicular to a camera view.    -   Take snap shot of the sheet, and correlate the position of the        dots to the camera image:

Dot1(x,y,1)←>pixel(x,y)

-   -   Place a different calibration sheet that contains known dots at        another different known distance (e.g. 2 meters), and        perpendicular to camera view.    -   Take another snapshot of the sheet, and correlate the position        of the dots to the camera image:

Dot2(x,y,2)←>pixel(x,y)

-   -   Smooth the dots and pixels to minimize digitization errors. By        smoothing the map using a global map function, step errors will        be eliminated and each pixel will be mapped to a cone space.    -   For each pixel, draw a line from Dot1(x,y,z) to Dot2(x, y, z)        defining a cone center where the camera can view.    -   One smoothing method is to apply a weighted filter for Dot1 and        Dot2. A weight filter can be used. In one example, the following        exemplary filter is applied.        -   1 2 1        -   2 4 2        -   1 2 1

Assuming Dot1_Left refers to the value of the dot on the left side ofDot1 and Dot1_Right refers to the value of the dot to the right of Dot1and Dot1_Upper refers to the dot above Dot1, for example, the resultingsmoothed Dot1 value is as follows:

1/16*(Dot1*4+Dot1_Left*2+Dot1_Right*2+Dot1_Upper*2+Dot1_Down*2+Dot1_UpperLeft+Dot1_UpperRight+Dot1_LowerLeft+Dot1_LowerRight)

Similarly, the resulting smoothed Dot2 value is as follows:

1/16* (Dot2*4+Dot2_Left*2+Dot2_Right*2+Dot2_Upper*2+Dot2_Down*2+Dot2_UpperLeft+Dot2_UpperRight+Dot2_LowerLeft+Dot2_LowerRight)

In another smoothing method, features from Dot1 sheet are mapped to asub pixel level and features of Dot2 sheet are mapped to a sub pixellevel and smooth them. To illustrate, Dot1 dot center (5, 5, 1) aremapped to pixel (1.05, 2.86), and Dot2 dot center (10, 10, 2) are mappedto pixel (1.15, 2.76). A predetermined correlation function is thenapplied.

FIG. 2B shows an exemplary calibration sheet having a plurality of dots.In this embodiment, the dots can be circular dots and square dots whichare interleaved among each other. The dots should be placed relativelyclose to each other and each dot size should not be too large, so we canhave as many dots as possible in one snapshot. However, the dots shouldnot be placed too close to each other and the dot size should not be toosmall, so they are not identifiable.

A module 54 monitors patient activity and generates a warning if thepatient has fallen. In one implementation, the system detects the speedof center of mass movement. If the center of mass movement is zero for apredetermined period, the patient is either sleeping or unconscious. Thesystem then attempts to signal the patient and receive confirmatorysignals indicating that the patient is conscious. If patient does notconfirm, then the system generates an alarm. For example, if the patienthas fallen, the system would generate an alarm signal that can be sentto friends, relatives or neighbors of the patient. Alternatively, athird party such as a call center can monitor the alarm signal. Besidesmonitoring for falls, the system performs video analysis of the patient.For example, during a particular day, the system can determine theamount of time for exercise, sleep, and entertainment, among others. Thenetwork of sensors in a patient's home can recognize ordinarypatterns—such as eating, sleeping, and greeting visitors—and to alertcaretakers to out-of-the-ordinary ones—such as prolonged inactivity orabsence. For instance, if the patient goes into the bathroom thendisappears off the sensor for 13 minutes and don't show up anywhere elsein the house, the system infers that patient had taken a bath or ashower. However, if a person falls and remains motionless for apredetermined period, the system would record the event and notify adesignated person to get assistance.

An exercise energy detection process (shown in FIG. 3) performs thefollowing operations:

-   -   Find floor space area    -   Define camera view background 3D scene    -   Calculate patient's key features    -   Detect patient movement and convert movement into energy        expenditure (such as calories burned).

In one implementation, pseudo-code for determining the floor space areais as follows:

-   -   1. Sample the camera view space by M by N, e.g. M=1000, N=500.    -   2. Calculate all sample points the 3D coordinates in room        coordinate system; where Z axis is pointing up. Refer to the 3D        detection for how to calculate 3D positions.    -   3. Find the lowest Z value point (Zmin)    -   4. Find all points whose Z values are less than Zmin+Ztol; where        Ztol is a user adjustable value, e.g. 2 inches.    -   5. If rooms have different elevation levels, then excluding the        lowest Z floor points, repeat step 2, 3 and 4 while keeping the        lowest Z is within Ztol2 of previous Z. In this example Ztol2=2        feet, which means the floor level difference should be within 2        feet.    -   6. Detect stairs by finding approximate same flat area but        within equal Z differences between them.    -   7. Optionally, additional information from the user can be used        to define floor space more accurately, especially in single        camera system where the coordinates are less accurate, e.g.:        -   a. Import the CAD file from constructors' blue prints.        -   b. Pick regions from the camera space to define the floor,            then use software to calculate its room coordinates.        -   c. User software to find all flat surfaces, e.g. floors,            counter tops, then user pick the ones, which are actually            floors and/or stairs.

In the implementation, pseudo-code for determining the camera viewbackground 3D scene is as follows:

-   -   1. With the same sample points, calculate x, y coordinates and        the Z depth and calculate 3D positions.    -   2. Determine background scene using one the following methods,        among others:        -   a. When there is nobody in the room.        -   b. Retrieve and update the previous calculated background            scene.        -   c. Continuous updating every sample point when the furthest            Z value was found, that is the background value.            In one implementation, pseudo-code for determining key            features of the patient is as follows:    -   1. Foreground objects can be extracted by comparing each sample        point's Z value to the background scene point's Z value, if it        is smaller, then it is on the foreground.    -   2. In normal condition, the feet/shoe can be detected by finding        the lowest Z point clouds close the floor in room space, its        color will be extracted.    -   3. In normal condition, the hair/hat can be detected by finding        the highest Z point clouds close the floor in room space, its        color will be extracted.    -   4. The rest of the features can be determined by searching from        either head or toe. E.g, hat, hair, face, eye, mouth, ear,        earring, neck, lipstick, moustache, jacket, limbs, belt, ring,        hand, etc.    -   5. The key dimension of features will be determined by        retrieving the historic stored data or recalculated, e.g., head        size, mouth width, arm length, leg length, waist, etc.    -   6. In abnormal conditions, features can be detected by detect        individual features then correlated them to different body        parts. E.g, if patient's skin is black, we can hardly get a        yellow or white face, by detecting eye and nose, we know which        part is the face, then we can detect other characteristics.

One embodiment can be used to detect falls—the pseudo-code for theembodiment is as follows:

-   -   1. The fall has to be detected in almost real time by tracking        movements of key features very quickly. E.g. if patient has        black hair/face, track the center of the black blob will know        roughly where his head move to.    -   2. Then the center of mass will be tracked, center of mass is        usually around belly button area, so the belt or borderline        between upper and lower body closed will be good indications.    -   3. Patient's fall always coupled with rapid deceleration of        center of mass. Software can adjust this threshold based on        patient age, height and physical conditions.    -   4. Then if the fall is accidental and patient has difficult to        get up, one or more of following will happen:        -   a. Patient will move very slowly to find support object to            get up.        -   b. Patient will wave hand to camera ask for help. To detect            this condition, the patient hand has to be detected first by            finding a blob of points with his skin color. Hand motion            can be tracked by calculate the motion of the center of the            points, if it swings left and right, it means patient is            waving to camera.        -   c. Patient is unconscious, motionless. To detect this            condition, extract the foreground object, calculate its            motion vectors, if it is within certain tolerance, it means            patient is not moving. In the mean time, test how long it            last, if it past a user defined time threshold, it means            patient is in great danger.

In one embodiment for fall detection, the system determines a patientfall-down as when the patient's knee, butt or hand is on the floor. Thefall action is defined a quick deceleration of center of mass, which isaround belly button area. An accidental fall action is defined when thepatient falls down with limited movement for a predetermined period.

The system monitors the patients' fall relative to a floor. In oneembodiment, the plan of the floor is specified in advance by thepatient. Alternatively, the system can automatically determine the floorlayout by examining the movement of the patient's feet and estimated thesurfaces touched by the feet as the floor.

In one embodiment with in-door positioning, the user can create afacsimile of the floor plan during initialization by walking around theperimeter of each room and recording his/her movement through thein-door positioning system and when complete, press a button to indicateto the system the type of room such as living room, bed room, bath room,among others. Also, the user can calibrate the floor level by sittingdown and then standing up (or vice versa) and allowing the accelerometerto sense the floor through the user motion. Periodically, the user canrecalibrate the floor plan and/or the floor level.

The system detects a patient exercise activity by detecting a center ofmass of an exemplary feature. Thus, the software can monitor the centerof one or more objects, for example the head and toe, the patient'sbelt, the bottom line of the shirt, or the top line of the pants.Movements relative to the center of mass can be converted into exercisemotion and energy expenditure can be determined.

In a fall detection warning embodiment, the detection of the patientexercise activity can be adjusted based on two thresholds:

a. Speed of deceleration of the center of mass.

b. The amount of time that the patient lies motionless on the floorafter the fall.

If the center of mass movement ceases to move for a predeterminedperiod, the system can generate the warning. In another embodiment,before generating the warning, the system can request the patient toconfirm that he or she does not need assistance. The confirmation can bein the form of a button that the user can press to override the warning.Alternatively, the confirmation can be in the form of a single utterancethat is then detected by a speech recognizer.

In another embodiment, the confirmatory signal is a patient gesture. Thepatient can nod his or her head to request help and can shake the headto cancel the help request. Alternatively, the patient can use aplurality of hand gestures to signal to the server 20 the actions thatthe patient desires.

By adding other detecting mechanism such as sweat detection, the systemcan know whether patient is uncomfortable or not. Other items that canbe monitored include chest movement (frequency and amplitude) and restlength when the patient sits still in one area, among others.

Besides monitoring for exercise activities and falls, the systemperforms video analysis of the patient. For example, during a particularday, the system can determine the amount of time for exercise, sleep,entertainment, among others. The network of sensors in a patient's homecan recognize ordinary patterns—such as eating, sleeping, and greetingvisitors—and to alert caretakers to out-of-the-ordinary ones—such asprolonged inactivity or absence. For instance, if the patient goes intothe bathroom then disappears off the camera 10 view for a predeterminedperiod and does not show up anywhere else in the house, the systeminfers that patient had taken a bath or a shower. However, if a personfalls and remains motionless for a predetermined period, the systemwould record the event and notify a designated person to get assistance.

In one embodiment, changes in the patient's skin color can be detectedby measuring the current light environment, properly calibrating colorspace between two photos, and then determining global color changebetween two states. Thus, when the patient's face turn red, based on theredness, a severity level warning is generated.

In another embodiment, changes in the patient's face are detected byanalyzing a texture distortion in the images. If the patient perspiresheavily, the texture will show small glisters, make-up smudges, orsweat/tear drippings. Another example is, when long stretched face willbe detected as texture distortion. Agony will show certain wrinkletexture patterns, among others.

The system can also utilize high light changes. Thus, when the patientsweats or changes facial appearance, different high light areas areshown, glisters reflect light and pop up geometry generates more highlight areas.

A module 62 analyzes facial changes such as facial asymmetries. Thechange will be detected by superimpose a newly acquired 3D anatomystructure to a historical (normal) 3D anatomy structure to detectface/eye sagging or excess stretch of facial muscles.

In one embodiment, the system determines a set of base 3D shapes, whichare a set of shapes which can represent extremes of certain facialeffects, e.g. frown, open mouth, smiling, among others. The rest of the3D face shape can be generated by blending/interpolating these baseshapes by applied different weight to each base shapes.

The base 3D shape can be captured using 1) a 3D camera such as camerasfrom Steinbichler, Genex Technology, Minolta 3D, Olympus 3D or 2) one ormore 2D camera with preset camera field of view (FOV) parameters. Tomake it more accurate, one or more special markers can be placed onpatient's face. For example, a known dimension square stick can beplaced on the forehead for camera calibration purposes.

Using the above 3D detection method, facial shapes are then extracted.The proper features (e.g. a wrinkle) will be detected and attached toeach base shape. These features can be animated or blended by changingthe weight of different shape(s). The proper features change can bedetected and determine what type of facial shape it will be.

Next, the system super-imposes two 3D facial shapes (historical ornormal facial shapes and current facial shapes). By matching featuresand geometry of changing areas on the face, closely blended shapes canbe matched and facial shape change detection can be performed. Byoverlaying the two shapes, the abnormal facial change such as saggingeyes or mouth can be detected.

The above processes are used to determine paralysis of specific regionsof the face or disorders in the peripheral or central nervous system(trigeminal paralysis; CVA, among others). The software also detectseyelid positions for evidence of ptosis (incomplete opening of one orboth eyelids) as a sign of innervation problems (CVA; Horner syndrome,for example). The software also checks eye movements for pathologicalconditions, mainly of neurological origin are reflected in aberrationsin eye movement. Pupil reaction is also checked for abnormal reaction ofthe pupil to light (pupil gets smaller the stronger the light) mayindicate various pathological conditions mainly of the nervous system.In patients treated for glaucoma pupillary status and motion pattern maybe important to the follow-up of adequate treatment. The software alsochecks for asymmetry in tongue movement, which is usually indicative ofneurological problems. Another check is neck veins: Engorgement of theneck veins may be an indication of heart failure or obstruction ofnormal blood flow from the head and upper extremities to the heart. Thesoftware also analyzes the face, which is usually a mirror of theemotional state of the observed subject. Fear, joy, anger, apathy areonly some of the emotions that can be readily detected, facialexpressions of emotions are relatively uniform regardless of age, sex,race, etc. This relative uniformity allows for the creation of computerprograms attempting to automatically diagnose people's emotional states.

Speech recognition is performed to determine a change in the form ofspeech (slurred speech, difficulties in the formation of words, forexample) may indicated neurological problems, such an observation canalso indicate some outward effects of various drugs or toxic agents.

In one embodiment shown in FIG. 4, a facial expression analysis processperforms the following operations:

Find floor space area

Define camera view background 3D scene

Calculate patient's key features

Extract face, arm, and leg objects

Detect face, arm and leg orientation during exercise

Detect movements associated with exercise and determine calorie burnedduring exercise

The first three steps are already discussed above. The patient's keyfeatures provide information on the location of the face, and once theface area has been determined, other features can be detected bydetecting relative position to each other and special characteristics ofthe features:

-   -   Eye: pupil can be detected by applying Chamfer matching        algorithm, by using stock pupil objects.    -   Hair: located on the top of the head, using previous stored hair        color to locate the hair point clouds.    -   Birthmarks, wrinkles and tattoos: pre store all these features        then use Chamfer matching to locate them.    -   Nose: nose bridge and nose holes usually show special        characteristics for detection, sometime depend on the view        angle, is side view, special silhouette will be shown.    -   Eye browse, Lips and Moustache: All these features have special        colors, e.g. red lipstick; and base shape, e.g. patient has no        expression with mouth closed. Software will locate these        features by color matching, then try to deform the base shape        based on expression, and match shape with expression, we can        detect objects and expression at the same time.    -   Arms and Legs, Teeth, earring, necklace: All these features can        be detected by color and style, which will give extra        information.        In one implementation, pseudo-code for detecting facial        orientation is as follows:    -   Detect forehead area    -   Use the previously determined features and superimpose them on        the base face model to detect a patient face orientation.        Depending on where patient is facing, for a side facing view,        silhouette edges will provide unique view information because        there is a one to one correspondent between the view and        silhouette shape.

Once the patient's face has been aligned to the right view, exemplarypseudo code to detect facial expression is as follows:

-   -   1. Detect shape change. The shape can be match by superimpose        different expression shapes to current shape, and judge by        minimum discrepancy. E.g. wide mouth open.    -   2. Detect occlusion. Sometime the expression can be detected by        occlusal of another objects, e.g., teeth show up means mouth is        open.    -   3. Detect texture map change. The expression can relate to        certain texture changes, if patient smile, certain wrinkles        patents will show up.    -   4. Detect highlight change. The expression can relate to certain        high light changes, if patient sweats or cries, different        highlight area will show up.

Similar operations can be done to detect arm and leg movements forenergy expenditure estimation.

Speech recognition can be performed in one embodiment to determine achange in the form of speech (slurred speech, difficulties in theformation of words, for example) may indicated neurological problems,such an observation can also indicate some outward effects of variousdrugs or toxic agents.

A module communicates with a third party such as the police department,a security monitoring center, or a call center. The module operates witha POTS telephone and can use a broadband medium such as DSL or cablenetwork if available. The module 80 requires that at least the telephoneis available as a lifeline support. In this embodiment, duplex soundtransmission is done using the POTS telephone network. The broadbandnetwork, if available, is optional for high resolution video and otheradvanced services transmission.

During operation, the module checks whether broadband network isavailable. If broadband network is available, the module 80 allows highresolution video, among others, to be broadcasted directly from theserver 20 to the third party or indirectly from the server 20 to theremote server 200 to the third party. In parallel, the module 80 allowssound to be transmitted using the telephone circuit. In this manner,high resolution video can be transmitted since sound data is separatelysent through the POTS network.

If broadband network is not available, the system relies on the POTStelephone network for transmission of voice and images. In this system,one or more images are compressed for burst transmission, and at therequest of the third party or the remote server 200, the telephone'ssound system is placed on hold for a brief period to allow transmissionof images over the POTS network. In this manner, existing POTS lifelinetelephone can be used to monitor patients. The resolution and quantityof images are selectable by the third party. Thus, using only thelifeline as a communication medium, the person monitoring the patientcan elect to only listen, to view one high resolution image with duplextelephone voice transmission, to view a few low resolution images, toview a compressed stream of low resolution video with digitized voice,among others.

During installation or while no live person in the scene, each camerawill capture its own environment objects and store it as backgroundimages, the software then detect the live person in the scene, changesof the live person, so only the portion of live person will be send tothe local server, other compression techniques will be applied, e.g.send changing file, balanced video streaming based on change.

The local server will control and schedule how the video/picture will besend, e.g. when the camera is view an empty room, no pictures will besent, the local server will also determine which camera is at the rightview, and request only the corresponding video be sent. The local serverwill determine which feature it is interested in looking at, e.g. faceand request only that portion be sent.

With predetermined background images and local server controlledstreaming, the system will enable higher resolution and more camerasystem by using narrower bandwidth.

Through this module, a police officer, a security agent, or a healthcareagent such as a physician at a remote location can engage, ininteractive visual communication with the patient. The patient's healthdata or audio-visual signal can be remotely accessed. The patient alsohas access to a video transmission of the third party. Should thepatient experience health symptoms requiring intervention and immediatecare, the health care practitioner at the central station may summonhelp from an emergency services provider. The emergency servicesprovider may send an ambulance, fire department personnel, familymember, or other emergency personnel to the patient's remote location.The emergency services provider may, perhaps, be an ambulance facility,a police station, the local fire department, or any suitable supportfacility.

Communication between the patient's remote location and the centralstation can be initiated by a variety of techniques. One method is bymanually or automatically placing a call on the telephone to thepatient's home or from the patient's home to the central station.

Alternatively, the system can ask a confirmatory question to the patientthrough text to speech software. The patient can be orally instructed bythe health practitioner to conduct specific physical activities such asspecific arm movements, walking, bending, among others. The examinationbegins during the initial conversation with the monitored subject. Anychanges in the spontaneous gestures of the body, arms and hands duringspeech as well as the fulfillment of nonspecific tasks are importantsigns of possible pathological events. The monitoring person caninstruct the monitored subject to perform a series of simple tasks whichcan be used for diagnosis of neurological abnormalities. Theseobservations may yield early indicators of the onset of a disease.

A network 100 such as the Internet receives images from the server 20and passes the data to one or more remote servers 200. The images aretransmitted from the server 200 over a secure communication link such asvirtual private network (VPN) to the remote server(s) 200.

In one embodiment where cameras are deployed, the server 200 collectsdata from a plurality of cameras and uses the 3D images technology todetermine if the patient needs help. The system can transmit video (liveor archived) to the friend, relative, neighbor, or call center for humanreview. At each viewer site, after a viewer specifies the correct URL tothe client browser computer, a connection with the server 200 isestablished and user identity authenticated using suitable password orother security mechanisms. The server 200 then retrieves the documentfrom its local disk or cache memory storage and transmits the contentover the network. In the typical scenario, the user of a Web browserrequests that a media stream file be downloaded, such as sending, inparticular, the URL of a media redirection file from a Web server. Themedia redirection file (MRF) is a type of specialized Hypertext MarkupLanguage (HTML) file that contains instructions for how to locate themultimedia file and in what format the multimedia file is in. The Webserver returns the MRF file to the user's browser program. The browserprogram then reads the MRF file to determine the location of the mediaserver containing one or more multimedia content files. The browser thenlaunches the associated media player application program and passes theMRF file to it. The media player reads the MRF file to obtain theinformation needed to open a connection to a media server, such as aURL, and the required protocol information, depending upon the type ofmedial content is in the file. The streaming media content file is thenrouted from the media server down to the user.

In the camera embodiment, the transactions between the server 200 andone of the remote servers 200 are detailed. The server 200 compares oneimage frame to the next image frame. If no difference exists, theduplicate frame is deleted to minimize storage space. If a differenceexists, only the difference information is stored as described in theJPEG standard. This operation effectively compresses video informationso that the camera images can be transmitted even at telephone modemspeed of 64 k or less. More aggressive compression techniques can beused. For example, patient movements can be clusterized into a group ofknown motion vectors, and patient movements can be described using a setof vectors. Only the vector data is saved. During view back, each vectoris translated into a picture object which is suitably rasterized. Theinformation can also be compressed as motion information.

Next, the server 200 transmits the compressed video to the remote server200. The server 200 stores and caches the video data so that multipleviewers can view the images at once since the server 200 is connected toa network link such as telephone line modem, cable modem, DSL modem, andATM transceiver, among others.

In one implementation, the servers 200 use RAID-5 striping and paritytechniques to organize data in a fault tolerant and efficient manner.The RAID (Redundant Array of Inexpensive Disks) approach is welldescribed in the literature and has various levels of operation,including RAID-5, and the data organization can achieve data storage ina fault tolerant and load balanced manner. RAID-5 provides that thestored data is spread among three or more disk drives, in a redundantmanner, so that even if one of the disk drives fails, the data stored onthe drive can be recovered in an efficient and error free manner fromthe other storage locations. This method also advantageously makes useof each of the disk drives in relatively equal and substantiallyparallel operations. Accordingly, if one has a six gigabyte clustervolume which spans three disk drives, each disk drive would beresponsible for servicing two gigabytes of the cluster volume. Each twogigabyte drive would be comprised of one-third redundant information, toprovide the redundant, and thus fault tolerant, operation required forthe RAID-5 approach. For additional physical security, the server can bestored in a Fire Safe or other secured box, so there is no chance toerase the recorded data, this is very important for forensic analysis.

The system can also monitor the patient's gait pattern and generatewarnings should the patient's gait patterns indicate that the patient islikely to fall. The system will detect patient skeleton structure,stride and frequency; and based on this information to judge whetherpatient has joint problem, asymmetrical bone structure, among others.The system can store historical gait information, and by overlayingcurrent structure to the historical (normal) gait information, gaitchanges can be detected. In the camera embodiment, an estimate of thegait pattern is done using the camera. In a camera-less embodiment, thegait can be sensed by providing a sensor on the floor and a sensor nearthe head and the variance in the two sensor positions are used toestimate gait characteristics.

The system also provides a patient interface 90 to assist the patient ineasily accessing information. In one embodiment, the patient interfaceincludes a touch screen; voice-activated text reading; one touchtelephone dialing; and video conferencing. The touch screen has largeicons that are pre-selected to the patient's needs, such as his or herfavorite web sites or application programs. The voice activated textreading allows a user with poor eye-sight to get information from thepatient interface 90. Buttons with pre-designated dialing numbers, orvideo conferencing contact information allow the user to call a friendor a healthcare provider quickly.

In one embodiment, medicine for the patient is tracked using radiofrequency identification (RFID) tags. In this embodiment, each drugcontainer is tracked through an RFID tag that is also a drug label. TheRF tag is an integrated circuit that is coupled with a mini-antenna totransmit data. The circuit contains memory that stores theidentification Code and other pertinent data to be transmitted when thechip is activated or interrogated using radio energy from a reader. Areader consists of an RF antenna, transceiver and a micro-processor. Thetransceiver sends activation signals to and receives identification datafrom the tag. The antenna may be enclosed with the reader or locatedoutside the reader as a separate piece. RFID readers communicatedirectly with the RFID tags and send encrypted usage data over thepatient's network to the server 200 and eventually over the Internet100. The readers can be built directly into the walls or the cabinetdoors.

In one embodiment, capacitively coupled RFID tags are used. Thecapacitive RFID tag includes a silicon microprocessor that can store 96bits of information, including the pharmaceutical manufacturer, drugname, usage instruction and a 40-bit serial number. A conductive carbonink acts as the tag's antenna and is applied to a paper substratethrough conventional printing means. The silicon chip is attached toprinted carbon-ink electrodes on the back of a paper label, creating alow-cost, disposable tag that can be integrated on the drug label. Theinformation stored on the drug labels is written in a Medicine MarkupLanguage (MML), which is based on the eXtensible Markup Language (XML).MML would allow all computers to communicate with any computer system ina similar way that Web servers read Hyper Text Markup Language (HTML),the common language used to create Web pages.

After receiving the medicine container, the patient places the medicinein a medicine cabinet, which is also equipped with a tag reader. Thissmart cabinet then tracks all medicine stored in it. It can track themedicine taken, how often the medicine is restocked and can let thepatient know when a particular medication is about to expire. At thispoint, the server 200 can order these items automatically. The server200 also monitors drug compliance, and if the patient does not removethe bottle to dispense medication as prescribed, the server 200 sends awarning to the healthcare provider.

The user's habits can be determined by the system. This is done bytracking location, ambulatory travel vectors and time in a database.Thus, if the user typically sleeps between 10 pm to 6 am, the locationwould reflect that the user's location maps to the bedroom between 10 pmand 6 am. In one exemplary system, the system builds a schedule of theuser's activity as follows:

Location Time Start Time End Heart Rate Bed room   10 pm   6 am 60-80 Gym room   6 am   7 am 90-120 Bath room   7 am 7:30 am 85-120 Diningroom 7:30 am 8:45 am 80-90  Home Office 8:45 am 11:30 am  85-100 . . .

The habit tracking is adaptive in that it gradually adjusts to theuser's new habits. If there are sudden changes, the system flags thesesudden changes for follow up. For instance, if the user spends threehours in the bathroom, the system prompts the third party (such as acall center) to follow up with the patient to make sure he or she doesnot need help.

In one embodiment, data driven analyzers may be used to track thepatient's habits. These data driven analyzers may incorporate a numberof models such as parametric statistical models, non-parametricstatistical models, clustering models, nearest neighbor models,regression methods, and engineered (artificial) neural networks. Priorto operation, data driven analyzers or models of the patient's habits orambulation patterns are built using one or more training sessions. Thedata used to build the analyzer or model in these sessions are typicallyreferred to as training data. As data driven analyzers are developed byexamining only training examples, the selection of the training data cansignificantly affect the accuracy and the learning speed of the datadriven analyzer. One approach used heretofore generates a separate dataset referred to as a test set for training purposes. The test set isused to avoid overfilling the model or analyzer to the training data.Overfitting refers to the situation where the analyzer has memorized thetraining data so well that it fails to fit or categorize unseen data.Typically, during the construction of the analyzer or model, theanalyzer's performance is tested against the test set. The selection ofthe analyzer or model parameters is performed iteratively until theperformance of the analyzer in classifying the test set reaches anoptimal point. At this point, the training process is completed. Analternative to using an independent training and test set is to use amethodology called cross-validation. Cross-validation can be used todetermine parameter values for a parametric analyzer or model for anon-parametric analyzer. In cross-validation, a single training data setis selected. Next, a number of different analyzers or models are builtby presenting different parts of the training data as test sets to theanalyzers in an iterative process. The parameter or model structure isthen determined on the basis of the combined performance of all modelsor analyzers. Under the cross-validation approach, the analyzer or modelis typically retrained with data using the determined optimal modelstructure.

In one embodiment, clustering operations are performed to detectpatterns in the data. In another embodiment, a neural network is used torecognize each pattern as the neural network is quite robust atrecognizing user habits or patterns. Once the treatment features havebeen characterized, the neural network then compares the input userinformation with stored templates of treatment vocabulary known by theneural network recognizer, among others. The recognition models caninclude a Hidden Markov Model (HMM), a dynamic programming model, aneural network, a fuzzy logic, or a template matcher, among others.These models may be used singly or in combination.

Dynamic programming considers all possible points within the permitteddomain for each value of i. Because the best path from the current pointto the next point is independent of what happens beyond that point.Thus, the total cost of [i(k), j(k)] is the cost of the point itselfplus the cost of the minimum path to it. Preferably, the values of thepredecessors can be kept in an M×N array, and the accumulated cost keptin a 2×N array to contain the accumulated costs of the immediatelypreceding column and the current column. However, this method requiressignificant computing resources. For the recognizer to find the optimaltime alignment between a sequence of frames and a sequence of nodemodels, it must compare most frames against a plurality of node models.One method of reducing the amount of computation required for dynamicprogramming is to use pruning Pruning terminates the dynamic programmingof a given portion of user habit information against a given treatmentmodel if the partial probability score for that comparison drops below agiven threshold. This greatly reduces computation.

Considered to be a generalization of dynamic programming, a hiddenMarkov model is used in the preferred embodiment to evaluate theprobability of occurrence of a sequence of observations O(1), O(2), . .. O(t), . . . , O(T), where each observation O(t) may be either adiscrete symbol under the VQ approach or a continuous vector. Thesequence of observations may be modeled as a probabilistic function ofan underlying Markov chain having state transitions that are notdirectly observable. In one embodiment, the Markov network is used tomodel a number of user habits and activities. The transitions betweenstates are represented by a transition matrix A=[a(i,j)]. Each a(i,j)term of the transition matrix is the probability of making a transitionto state j given that the model is in state i. The output symbolprobability of the model is represented by a set of functions B=[b(j)(O(t)], where the b(j) (O(t) term of the output symbol matrix is theprobability of outputting observation O(t), given that the model is instate j. The first state is always constrained to be the initial statefor the first time frame of the utterance, as only a prescribed set ofleft to right state transitions are possible. A predetermined finalstate is defined from which transitions to other states cannot occur.Transitions are restricted to reentry of a state or entry to one of thenext two states. Such transitions are defined in the model as transitionprobabilities. Although the preferred embodiment restricts the flowgraphs to the present state or to the next two states, one skilled inthe art can build an HMM model without any transition restrictions,although the sum of all the probabilities of transitioning from anystate must still add up to one. In each state of the model, the currentfeature frame may be identified with one of a set of predefined outputsymbols or may be labeled probabilistically. In this case, the outputsymbol probability b(j) O(t) corresponds to the probability assigned bythe model that the feature frame symbol is O(t). The model arrangementis a matrix A=[a(i,j)] of transition probabilities and a technique ofcomputing B=b(j) O(t), the feature frame symbol probability in state j.The Markov model is formed for a reference pattern from a plurality ofsequences of training patterns and the output symbol probabilities aremultivariate Gaussian function probability densities. The patient habitinformation is processed by a feature extractor. During learning, theresulting feature vector series is processed by a parameter estimator,whose output is provided to the hidden Markov model. The hidden Markovmodel is used to derive a set of reference pattern templates, eachtemplate representative of an identified pattern in a vocabulary set ofreference treatment patterns. The Markov model reference templates arenext utilized to classify a sequence of observations into one of thereference patterns based on the probability of generating theobservations from each Markov model reference pattern template. Duringrecognition, the unknown pattern can then be identified as the referencepattern with the highest probability in the likelihood calculator. TheHMM template has a number of states, each having a discrete value.However, because treatment pattern features may have a dynamic patternin contrast to a single value. The addition of a neural network at thefront end of the HMM in an embodiment provides the capability ofrepresenting states with dynamic values. The input layer of the neuralnetwork comprises input neurons. The outputs of the input layer aredistributed to all neurons in the middle layer Similarly, the outputs ofthe middle layer are distributed to all output states, which normallywould be the output layer of the neuron. However, each output hastransition probabilities to itself or to the next outputs, thus forminga modified HMM. Each state of the thus formed HMM is capable ofresponding to a particular dynamic signal, resulting in a more robustHMM. Alternatively, the neural network can be used alone withoutresorting to the transition probabilities of the HMM architecture.

The system allows patients to conduct a low-cost, comprehensive,real-time monitoring of their vital parameters such as ambulation andfalls. Information can be viewed using an Internet-based website, apersonal computer, or simply by viewing a display on the monitor. Datameasured several times each day provide a relatively comprehensive dataset compared to that measured during medical appointments separated byseveral weeks or even months. This allows both the patient and medicalprofessional to observe trends in the data, such as a gradual increaseor decrease in blood pressure, which may indicate a medical condition.The invention also minimizes effects of white coat syndrome since themonitor automatically makes measurements with basically no discomfort;measurements are made at the patient's home or work, rather than in amedical office.

The wearable appliance is small, easily worn by the patient duringperiods of exercise or day-to-day activities, and non-invasivelymeasures blood pressure can be done in a matter of seconds withoutaffecting the patient. An on-board or remote processor can analyze thetime-dependent measurements to generate statistics on a patient's bloodpressure (e.g., average pressures, standard deviation, beat-to-beatpressure variations) that are not available with conventional devicesthat only measure systolic and diastolic blood pressure at isolatedtimes.

The wearable appliance provides an in-depth, cost-effective mechanism toevaluate a patient's health condition. Certain cardiac conditions can becontrolled, and in some cases predicted, before they actually occur.Moreover, data from the patient can be collected and analyzed while thepatient participates in their normal, day-to-day activities.

Software programs associated with the Internet-accessible website,secondary software system, and the personal computer analyze the bloodpressure, and heart rate, and pulse oximetry values to characterize thepatient's cardiac condition. These programs, for example, may provide areport that features statistical analysis of these data to determineaverages, data displayed in a graphical format, trends, and comparisonsto doctor-recommended values.

When the appliance cannot communicate with the mesh network, theappliance simply stores information in memory and continues to makemeasurements. The watch component automatically transmits all the storedinformation (along with a time /date stamp) when it comes in proximityto the wireless mesh network, which then transmits the informationthrough the wireless network.

In one embodiment, the server provides a web services that communicatewith third party software through an interface. To generate vitalparameters such as blood pressure information for the web servicessoftware interface, the patient continuously wears the blood-pressuremonitor for a short period of time, e.g. one to two weeks after visitinga medical professional during a typical ‘check up’ or after signing upfor a short-term monitoring program through the website. In this case,the wearable device such as the watch measures mobility through theaccelerometer and blood pressure in a near-continuous, periodic mannersuch as every fifteen minutes. This information is then transmitted overthe mesh network to a base station that communicates over the Internetto the server.

To view information sent from the blood-pressure monitor and falldetector on the wearable appliance, the patient or an authorized thirdparty such as family members, emergency personnel, or medicalprofessional accesses a patient user interface hosted on the web server200 through the Internet 100 from a remote computer system. The patientinterface displays vital information such as ambulation, blood pressureand related data measured from a single patient. The system may alsoinclude a call center, typically staffed with medical professionals suchas doctors, nurses, or nurse practioners, whom access a care-providerinterface hosted on the same website on the server 200. Thecare-provider interface displays vital data from multiple patients.

The wearable appliance has an indoor positioning system and processesthese signals to determine a location (e.g., latitude, longitude, andaltitude) of the monitor and, presumably, the patient. This locationcould be plotted on a map by the server, and used to locate a patientduring an emergency, e.g. to dispatch an ambulance.

In one embodiment, the web page hosted by the server 200 includes aheader field that lists general information about the patient (e.g.name, age, and ID number, general location, and information concerningrecent measurements); a table that lists recently measured bloodpressure data and suggested (i.e. doctor-recommended) values of thesedata; and graphs that plot the systolic and diastolic blood pressuredata in a time-dependent manner. The header field additionally includesa series of tabs that each link to separate web pages that include,e.g., tables and graphs corresponding to a different data measured bythe wearable device such as calorie consumption/dissipation, ambulationpattern, sleeping pattern, heart rate, pulse oximetry, and temperature.The table lists a series of data fields that show running average valuesof the patient's daily, monthly, and yearly vital parameters. The levelsare compared to a series of corresponding ‘suggested’ values of vitalparameters that are extracted from a database associated with the website. The suggested values depend on, among other things, the patient'sage, sex, and weight. The table then calculates the difference betweenthe running average and suggested values to give the patient an idea ofhow their data compares to that of a healthy patient. The web softwareinterface may also include security measures such as authentication,authorization, encryption, credential presentation, and digitalsignature resolution. The interface may also be modified to conform toindustry-mandated, XML schema definitions, while being backwardscompatible' with any existing XML schema definitions.

The system provides for self-registration of appliances by the user.Data can be synchronized between the Repository and appliance(s) via thebase station 20. The user can preview the readings received from theappliance(s) and reject erroneous readings. The user or treatingprofessional can set up the system to generate alerts against receiveddata, based on pre-defined parameters. The system can determine trendsin received data, based on user defined parameters.

Appliance registration is the process by which a patient monitoringappliance is associated with one or more users of the system. Thismechanism is also used when provisioning appliances for a user by athird party, such as a clinician (or their respective delegate). In oneimplementation, the user (or delegate) logs into the portal to selectone or more appliances and available for registration. In turn, the basestation server 20 broadcasts a query to all nodes in the mesh network toretrieve identification information for the appliance such asmanufacturer information, appliance model information, appliance serialnumber and optionally a hub number (available on hub packaging). Theuser may register more than one appliance at this point. The systemoptionally sets up a service subscription for appliance(s) usage. Thisincludes selecting service plans and providing payment information. Theappliance(s) are then associated with this user's account and a controlfile with appliance identification information is synchronized betweenthe server 200 and the base station 20 and each appliance oninitialization. In one embodiment, each appliance 8 transmits data tothe base station 20 in an XML format for ease of interfacing and iseither kept encrypted or in a non-readable format on the base station 20for security reasons.

The base station 20 frequently collects and synchronizes data from theappliances 8. The base station 20 may use one of various transportationmethods to connect to the repository on the server 200 using a PC asconduit or through a connection established using an embedded modem(connected to a phone line), a wireless router (DSL or cable wirelessrouter), a cellular modem, or another network-connected appliance (suchas, but not limited to, a web-phone, video-phone, embedded computer, PDAor handheld computer).

In one embodiment, users may set up alerts or reminders that aretriggered when one or more reading meet a certain set of conditions,depending on parameters defined by the user. The user chooses thecondition that they would like to be alerted to and by providing theparameters (e.g. threshold value for the reading) for alert generation.Each alert may have an interval which may be either the number of datapoints or a time duration in units such as hours, days, weeks or months.The user chooses the destination where the alert may be sent. Thisdestination may include the user's portal, e-mail, pager, voice-mail orany combination of the above.

Trends are determined by applying mathematical and statistical rules(e.g. moving average and deviation) over a set of reading values. Eachrule is configurable by parameters that are either automaticallycalculated or are set by the user.

The user may give permission to others as needed to read or edit theirpersonal data or receive alerts. The user or clinician could have a listof people that they want to monitor and have it show on their “MyAccount” page, which serves as a local central monitoring station in oneembodiment. Each person may be assigned different access rights whichmay be more or less than the access rights that the patient has. Forexample, a doctor or clinician could be allowed to edit data for exampleto annotate it, while the patient would have read-only privileges forcertain pages. An authorized person could set the reminders and alertsparameters with limited access to others. In one embodiment, the basestation server 20 serves a web page customized by the user or the user'srepresentative as the monitoring center that third parties such asfamily, physicians, or caregivers can log in and access information. Inanother embodiment, the base station 20 communicates with the server 200at a call center so that the call center provides all services. In yetanother embodiment, a hybrid solution where authorized representativescan log in to the base station server 20 access patient informationwhile the call center logs into both the server 200 and the base stationserver 20 to provide complete care services to the patient.

The server 200 may communicate with a business process outsourcing (BPO)company or a call center to provide central monitoring in an environmentwhere a small number of monitoring agents can cost effectively monitormultiple people 24 hours a day. A call center agent, a clinician or anursing home manager may monitor a group or a number of users via asummary “dashboard” of their readings data, with ability to drill-downinto details for the collected data. A clinician administrator maymonitor the data for and otherwise administer a number of users of thesystem. A summary “dashboard” of readings from all Patients assigned tothe Administrator is displayed upon log in to the Portal by theAdministrator. Readings may be color coded to visually distinguishnormal vs. readings that have generated an alert, along with descriptionof the alert generated. The Administrator may drill down into thedetails for each Patient to further examine the readings data, viewcharts etc. in a manner similar to the Patient's own use of the system.The Administrator may also view a summary of all the appliancesregistered to all assigned Patients, including but not limited to allappliance identification information. The Administrator has access onlyto information about Patients that have been assigned to theAdministrator by a Super Administrator. This allows for segmenting theentire population of monitored Patients amongst multiple Administrators.The Super Administrator may assign, remove and/or reassign Patientsamongst a number of Administrators.

In one embodiment, a patient using an Internet-accessible computer andweb browser, directs the browser to an appropriate URL and signs up fora service for a short-term (e.g., 1 month) period of time. The companyproviding the service completes an accompanying financial transaction(e.g. processes a credit card), registers the patient, and ships thepatient a wearable appliance for the short period of time. Theregistration process involves recording the patient's name and contactinformation, a number associated with the monitor (e.g. a serialnumber), and setting up a personalized website. The patient then usesthe monitor throughout the monitoring period, e.g. while working,sleeping, and exercising. During this time the monitor measures datafrom the patient and wirelessly transmits it through the channel to adata center. There, the data are analyzed using software running oncomputer servers to generate a statistical report. The computer serversthen automatically send the report to the patient using email, regularmail, or a facsimile machine at different times during the monitoringperiod. When the monitoring period is expired, the patient ships thewearable appliance back to the monitoring company.

Different web pages may be designed and accessed depending on theend-user. As described above, individual users have access to web pagesthat only their ambulation and blood pressure data (i.e., the patientinterface), while organizations that support a large number of patients(nursing homes or hospitals) have access to web pages that contain datafrom a group of patients using a care-provider interface. Otherinterfaces can also be used with the web site, such as interfaces usedfor: insurance companies, members of a particular company, clinicaltrials for pharmaceutical companies, and e-commerce purposes. Vitalpatient data displayed on these web pages, for example, can be sortedand analyzed depending on the patient's medical history, age, sex,medical condition, and geographic location. The web pages also support awide range of algorithms that can be used to analyze data once they areextracted from the data packets. For example, an instant message oremail can be sent out as an ‘alert’ in response to blood pressureindicating a medical condition that requires immediate attention.Alternatively, the message could be sent out when a data parameter (e.g.systolic blood pressure) exceeds a predetermined value. In some cases,multiple parameters (e.g., fall detection, positioning data, and bloodpressure) can be analyzed simultaneously to generate an alert message.In general, an alert message can be sent out after analyzing one or moredata parameters using any type of algorithm. These algorithms range fromthe relatively simple (e.g., comparing blood pressure to a recommendedvalue) to the complex (e.g., predictive medical diagnoses using ‘datamining’ techniques). In some cases data may be ‘fit’ using algorithmssuch as a linear or non-linear least-squares fitting algorithm.

In one embodiment, a physician, other health care practitioner, oremergency personnel is provided with access to patient medicalinformation through the server 200. In one embodiment, if the wearableappliance detects that the patient needs help, or if the patient decideshelp is needed, the system can call his or her primary care physician.If the patient is unable to access his or her primary care physician (oranother practicing physician providing care to the patient) a call fromthe patient is received, by an answering service or a call centerassociated with the patient or with the practicing physician. The callcenter determines whether the patient is exhibiting symptoms of anemergency condition by polling vital patient information generated bythe wearable device, and if so, the answering service contacts 911emergency service or some other emergency service. The call center canreview falls information, blood pressure information, and other vitalinformation to determine if the patient is in need of emergencyassistance. If it is determined that the patient in not exhibitingsymptoms of an emergent condition, the answering service may thendetermine if the patient is exhibiting symptoms of a non-urgentcondition. If the patient is exhibiting symptoms of a non-urgentcondition, the answering service will inform the patient that he or shemay log into the server 200 for immediate information on treatment ofthe condition. If the answering service determines that the patient isexhibiting symptoms that are not related to a non-urgent condition, theanswering service may refer the patient to an emergency room, a clinic,the practicing physician (when the practicing physician is available)for treatment.

In another embodiment, the wearable appliance permits direct access tothe call center when the user pushes a switch or button on theappliance, for instance. In one implementation, telephones and switchingsystems in call centers are integrated with the home mesh network toprovide for, among other things, better routing of telephone calls,faster delivery of telephone calls and associated information, andimproved service with regard to client satisfaction throughcomputer-telephony integration (CTI). CTI implementations of variousdesign and purpose are implemented both within individual call-centersand, in some cases, at the telephone network level. For example,processors running CTI software applications may be linked to telephoneswitches, service control points (SCPs), and network entry points withina public or private telephone network. At the call-center level,CTI-enhanced processors, data servers, transaction servers, and thelike, are linked to telephone switches and, in some cases, to similarCTI hardware at the network level, often by a dedicated digital link CTIprocessors and other hardware within a call-center is commonly referredto as customer premises equipment (CPE). It is the CTI processor andapplication software is such centers that provides computer enhancementto a call center. In a CTI-enhanced call center, telephones at agentstations are connected to a central telephony switching apparatus, suchas an automatic call distributor (ACD) switch or a private branchexchange (PBX). The agent stations may also be equipped with computerterminals such as personal computer/video display unit's (PC/VDU's) sothat agents manning such stations may have access to stored data as wellas being linked to incoming callers by telephone equipment. Suchstations may be interconnected through the PC/VDUs by a local areanetwork (LAN). One or more data or transaction servers may also beconnected to the LAN that interconnects agent stations. The LAN is, inturn, typically connected to the CTI processor, which is connected tothe call switching apparatus of the call center.

When a call from a patient arrives at a call center, whether or not thecall has been pre-processed at an SCP, the telephone number of thecalling line and the medical record are made available to the receivingswitch at the call center by the network provider. This service isavailable by most networks as caller-ID information in one of severalformats such as Automatic Number Identification (ANI). Typically thenumber called is also available through a service such as Dialed NumberIdentification Service (DNIS). If the call center is computer-enhanced(CTI), the phone number of the calling party may be used as a key toaccess additional medical and/or historical information from a customerinformation system (CIS) database at a server on the network thatconnects the agent workstations. In this manner information pertinent toa call may be provided to an agent, often as a screen pop on the agent'sPC/VDU.

The call center enables any of a first plurality of physician or healthcare practitioner terminals to be in audio communication over thenetwork with any of a second plurality of patient wearable appliances.The call center will route the call to a physician or other health carepractitioner at a physician or health care practitioner terminal andinformation related to the patient (such as an electronic medicalrecord) will be received at the physician or health care practitionerterminal via the network. The information may be forwarded via acomputer or database in the practicing physician's office or by acomputer or database associated with the practicing physician, a healthcare management system or other health care facility or an insuranceprovider. The physician or health care practitioner is then permitted toassess the patient, to treat the patient accordingly, and to forwardupdated information related to the patient (such as examination,treatment and prescription details related to the patient's visit to thepatient terminal) to the practicing physician via the network 200.

In one embodiment, the system informs a patient of a practicingphysician of the availability of the web services and referring thepatient to the web site upon agreement of the patient. A call from thepatient is received at a call center. The call center enables physiciansto be in audio communication over the network with any patient wearableappliances, and the call is routed to an available physician at one ofthe physician so that the available physician may carry on a two-wayconversation with the patient. The available physician is permitted tomake an assessment of the patient and to treat the patient. The systemcan forward information related to the patient to a health caremanagement system associated with the physician. The health caremanagement system may be a healthcare management organization, a pointof service health care system, or a preferred provider organization. Thehealth care practitioner may be a nurse practitioner or an internist.

The available health care practitioner can make an assessment of thepatient and to conduct an examination of the patient over the network,including optionally by a visual study of the patient. The system canmake an assessment in accordance with a protocol. The assessment can bemade in accordance with a protocol stored in a database and/or making anassessment in accordance with the protocol may include displaying inreal time a relevant segment of the protocol to the available physicianSimilarly, permitting the physician to prescribe a treatment may includepermitting the physician to refer the patient to a third party fortreatment and/or referring the patient to a third party for treatmentmay include referring the patient to one or more of a primary carephysician, specialist, hospital, emergency room, ambulance service orclinic. Referring the patient to a third party may additionally includecommunicating with the third party via an electronic link included in arelevant segment of a protocol stored in a protocol database resident ona digital storage medium and the electronic link may be a hypertextlink. When a treatment is being prescribed by a physician, the systemcan communicate a prescription over the network to a pharmacy and/orcommunicating the prescription over the network to the pharmacy mayinclude communicating to the pharmacy instructions to be given to thepatient pertaining to the treatment of the patient. Communicating theprescription over the network to the pharmacy may also includecommunicating the prescription to the pharmacy via a hypertext linkincluded in a relevant segment of a protocol stored in a databaseresident on a digital storage medium. In accordance with another relatedembodiment, permitting the physician to conduct the examination may beaccomplished under conditions such that the examination is conductedwithout medical instruments at the patient terminal where the patient islocated.

In another embodiment, a system for delivering medical examination,diagnosis, and treatment services from a physician to a patient over anetwork includes a first plurality of health care practitioners at aplurality of terminals, each of the first plurality of health carepractitioner terminals including a display device that shows informationcollected by the wearable appliances and a second plurality of patientterminals or wearable appliances in audiovisual communication over anetwork with any of the first plurality of health care practitionerterminals. A call center is in communication with the patient wearableappliances and the health care practitioner terminals, the call centerrouting a call from a patient at one of the patient terminals to anavailable health care practitioner at one of the health carepractitioner terminals, so that the available health care practitionermay carry on a two-way conversation with the patient. A protocoldatabase resident on a digital storage medium is accessible to each ofthe health care practitioner terminals. The protocol database contains aplurality of protocol segments such that a relevant segment of theprotocol may be displayed in real time on the display device of thehealth care practitioner terminal of the available health carepractitioner for use by the available health care practitioner in makingan assessment of the patient. Tthe relevant segment of the protocoldisplayed in real time on the display device of the health carepractitioner terminal may include an electronic link that establishescommunication between the available health care practitioner and a thirdparty and the third party may be one or more of a primary carephysician, specialist, hospital, emergency room, ambulance service,clinic or pharmacy.

In accordance with other related embodiment, the patient wearableappliance may include establish a direct connection to the call centerby pushing a button on the appliance. Further, the protocol database maybe resident on a server that is in communication with each of the healthcare practitioner terminals and each of the health care practitionerterminals may include a local storage device and the protocol databaseis replicated on the local storage device of one or more of thephysician terminals.

In another embodiment, a system for delivering medical examination,diagnosis, and treatment services from a physician to a patient over anetwork includes a first plurality of health care practitionerterminals, each of the first plurality of health care practitionerterminals including a display device and a second plurality of patientterminals in audiovisual communication over a network with any of thefirst plurality of health care practitioner terminals. Each of thesecond plurality of patient terminals includes a camera having pan, tiltand zoom modes, such modes being controlled from the first plurality ofhealth care practitioner terminals. A call center is in communicationwith the patient terminals and the health care practitioner terminalsand the call center routes a call from a patient at one of the patientterminals to an available health care practitioner at one of the healthcare practitioner terminals, so that the available health carepractitioner may carry on a two-way conversation with the patient andvisually observe the patient.

In one embodiment, the information is store in a secure environment,with security levels equal to those of online banking, social securitynumber input, and other confidential information. Conforming to HealthInsurance Portability and Accountability Act (HIPAA) requirements, thesystem creates audit trails, requires logins and passwords, and providesdata encryption to ensure the patient information is private and secure.The HIPAA privacy regulations ensure a national floor of privacyprotections for patients by limiting the ways that health plans,pharmacies, hospitals and other covered entities can use patients'personal medical information. The regulations protect medical recordsand other individually identifiable health information, whether it is onpaper, in computers or communicated orally.

Due to its awareness of the patient's position, the server 200 canoptionally control a mobility assistance device such as a smart cane orrobot. The robotic smart cane sends video from its camera to the server20, which in turn coordinates the position of the robot, as determinedby the cameras 10 mounted in the home as well as the robot camera. Therobot position, as determined by the server 20, is then transmitted tothe robot for navigation. The robot has a frame with an extended handle.The handle includes handle sensors mounted thereon to detect the forceplaces on each handle to receive as input the movement desired by thepatient. In one embodiment, the robot has a control navigation systemthat accepts patient command as well as robot self-guidance command. Themobility is a result of give-and-take between the patient'sself-propulsion and the walker's automated reactions. Thus, when thepatient moves the handle to the right, the robot determines that thepatient is interested in turning and actuates the drive systemsappropriately. However, if the patient is turning into an obstacle, asdetermined by the cameras and the server 20, the drive system providesgentle resistance that tells the patient of an impending collision.

If, for example, a patient does not see a coffee table ahead, the walkerwill detect it, override the patient's steering to avoid it, and therebyprevent a possible fall. Onboard software processes the data from 180degrees of approaching terrain and steers the front wheel towardopenings and away from obstacles.

The control module executes software that enables the robot to movearound its environment safely. The software performs localization,mapping, path planning and obstacle avoidance. In one embodiment, imagesfrom a plurality of wall-mounted cameras 10 are transmitted to theserver 20. The server 20 collects images of the robot and triangulatesthe robot position by cross-referencing the images. The information isthen correlated with the image from the robot-mounted camera and opticalencoders that count the wheel rotations to calculate traveled distancefor range measurement. In this process, a visual map of unique“landmarks” created as the robot moves along its path is annotated withthe robot's position to indicate the position estimate of the landmark.The current image, seen from the robot, is compared with the images inthe database to find matching landmarks. Such matches are used to updatethe position of the robot according to the relative position of thematching landmark. By repeatedly updating the position of landmarksbased on new data, the software incrementally improves the map bycalculating more accurate estimates for the position of the landmarks.An improved map results in more accurate robot position estimates.Better position estimates contribute to better estimates for thelandmark positions and so on. If the environment changes so much thatthe robot no longer recognizes previous landmarks, the robotautomatically updates the map with new landmarks. Outdated landmarksthat are no longer recognized can easily be deleted from the map bysimply determining if they were seen or matched when expected.

Using the obstacle avoidance algorithm, the robot generates correctivemovements to avoid obstacles not represented in the path planner such asopen/closed doors, furniture, people, and more. The robot rapidlydetects obstacles using its sensors and controls its speed and headingto avoid obstacles.

The hazard avoidance mechanisms provide a reflexive response tohazardous situations to insure the robot's safety and guarantee that itdoes not damage itself or the environment. Mechanisms for hazardavoidance include collision detection using not one but a complementaryset of sensors and techniques. For instance, collision avoidance can beprovided using contact sensing, motor load sensing, and vision. Thecombination of multiple sources for collision detection guarantees safecollision avoidance. Collision detection provides a last resort fornegotiating obstacles in case obstacle avoidance fails to do so in thefirst place, which can be caused by moving objects or software andhardware failures.

If the walker is in motion (as determined by the wheel encoder), theforce applied to the brake pads is inversely proportional to thedistance to obstacles. If the walker is stopped, the brakes should befully applied to provide a stable base on which the patient can rest.When the walker is stopped and the patient wishes to move again, thebrakes should come off slowly to prevent the walker from lurchingforward

The walker should mostly follow the patient's commands, as this iscrucial for patient acceptance. For the safety braking and the safetybraking and steering control systems, the control system only influencesthe motion when obstacles or cliffs are near the patient. In otherwords, the walker is, typically, fully patient controlled. For all othersituations, the control system submits to the patient's desire. Thisdoes not mean that the control system shuts down, or does not providethe usual safety features. In fact, all of the control systems fall backon their emergency braking to keep the patient safe. When the controlsystem has had to brake to avoid an obstacle or has given up trying tolead the patient on a particular path, the patient must disengage thebrakes (via a pushbutton) or re-engage the path following (again via apushbutton) to regain control or allow collaboration again. This letsthe patient select the walker's mode manually when they disagree withthe control system's choices.

FIG. 5 shows an exemplary process to monitor patient. First, the processsets up mesh network appliances (1000). Next, the process determinespatient position using in-door positioning system (1002). The processthen determines patient movement using accelerometer output (1004).Sharp accelerations may be used to indicate fall. Further, the z axisaccelerometer changes can indicate the height of the appliance from thefloor and if the height is near zero, the system infers that the patienthad fallen. The system can also determine vital parameter includingpatient heart rate (1006). The system determines if patient needsassistance based on in-door position, fall detection and vital parameter(1008). If a fall is suspected, the system confirms the fall bycommunicating with the patient prior to calling a third party such asthe patient's physician, nurse, family member, 911, 511, 411, or a paidcall center to get assistance for the patient (1010). If confirmed or ifthe patient is non-responsive, the system contacts the third party andsends voice over mesh network to appliance on the patient to allow oneor more third parties to talk with the patient (1012). If needed, thesystem calls and/or conferences emergency personnel into the call(1014).

In one embodiment, if the patient is outside of the mesh network rangesuch as when the user is traveling away from his/her home, the systemcontinuously records information into memory until the home mesh networkis reached or until the monitoring appliance reaches an interne accesspoint. While the wearable appliance is outside of the mesh networkrange, the device searches for a cell phone with an expansion cardplugged into a cell phone expansion slot such as the SDIO slot. If thewearable appliance detects a cell phone that is mesh network compatible,the wearable appliance communicates with the cell phone and providesinformation to the server 200 using the cellular connection. In oneembodiment, a Zigbee SDIO card from C-guys, Inc., enablesdevice-to-device communications for PDAs and smart phones. C-guys'ZigBee SDIO card includes the company's CG-100 SDIO applicationinterface controller, which is designed to convert an application signalto an SD signal (or vice versa). The ZigBee card can provide signalranges of up to 10 m in the 2.4 GHz band and data rates of up to 200kbps. The card has peer-to-peer communications mode and supports directapplication to PDAs or any SD supported hand-held cell phones. In thisembodiment, the PDA or cell phone can provide a GPS position informationinstead of the indoor position information generated by the mesh networkappliances 8. The cell phone GPS position information, accelerometerinformation and vital information such as heart rate information istransmitted using the cellular channel to the server 200 for processingas is normal. In another embodiment where the phone works through WiFi(802.11) or WiMAX (802.16) or ultra-wideband protocol instead of thecellular protocol, the wearable appliance can communicate over theseprotocols using a suitable mesh network interface to the phone. Ininstances where the wearable appliance is outside of its home base and adangerous condition such as a fall is detected, the wearable appliancecan initiate a distress call to the authorized third party usingcellular, WiFi, WiMAX, or UWB protocols as is available.

FIG. 6A shows a portable embodiment of the present invention where thevoice recognizer is housed in a wrist-watch. As shown in FIG. 6, thedevice includes a wrist-watch sized case 1380 supported on a wrist band1374. The case 1380 may be of a number of variations of shape but can beconveniently made a rectangular, approaching a box-like configuration.The wrist-band 1374 can be an expansion band or a wristwatch strap ofplastic, leather or woven material. The processor or CPU of the wearableappliance is connected to a radio frequency (RF) transmitter/receiver(such as a Bluetooth device, a Zigbee device, a WiFi device, a WiMAXdevice, or an 802.X transceiver, among others.

In one embodiment, the back of the device is a conductive metalelectrode 1381 that in conjunction with a second electrode 1383 mountedon the wrist band 1374, enables differential EKG or ECG to be measured.The electrical signal derived from the electrodes is typically lmVpeak-peak. In one embodiment where only one electrode 1381 or 1383isavailable, an amplification of about 1000 is necessary to render thissignal usable for heart rate detection. In the embodiment withelectrodes 1381 and 1383 available, a differential amplifier is used totake advantage of the identical common mode signals from the EKG contactpoints, the common mode noise is automatically cancelled out using amatched differential amplifier. In one embodiment, the differentialamplifier is a Texas Instruments INA321 instrumentation amplifier thathas matched and balanced integrated gain resistors. This device isspecified to operate with a minimum of 2.7V single rail power supply.The INA321 provides a fixed amplification of 5× for the EKG signal. Withits CMRR specification of 94 dB extended up to 3 KHz the INA321 rejectsthe common mode noise signals including the line frequency and itsharmonics. The quiescent current of the INA321 is 40 mA and the shutdown mode current is less than 1 mA. The amplified EKG signal isinternally fed to the on chip analog to digital converter. The ADCsamples the EKG signal with a sampling frequency of 512 Hz. Precisesampling period is achieved by triggering the ADC conversions with atimer that is clocked from a 32.768 kHz low frequency crystaloscillator. The sampled EKG waveform contains some amount of superimposed line frequency content. This line frequency noise is removed bydigitally filtering the samples. In one implementation, a 17-tap lowpass FIR filter with pass band upper frequency of 6 Hz and stop bandlower frequency of 30 Hz is implemented in this application. The filtercoefficients are scaled to compensate the filter attenuation and provideadditional gain for the EKG signal at the filter output. This adds up toa total amplification factor of greater than 1000× for the EKG signal.

The wrist band 1374 can also contain other electrical devices such asultrasound transducer, optical transducer or electromagnetic sensors,among others. In one embodiment, the transducer is an ultrasonictransducer that generates and transmits an acoustic wave upon commandfrom the CPU during one period and listens to the echo returns during asubsequent period. In use, the transmitted bursts of sonic energy arescattered by red blood cells flowing through the subject's radialartery, and a portion of the scattered energy is directed back towardthe ultrasonic transducer 84. The time required for the return energy toreach the ultrasonic transducer varies according to the speed of soundin the tissue and according to the depth of the artery. Typical transittimes are in the range of 6 to 7 microseconds. The ultrasonic transduceris used to receive the reflected ultrasound energy during the dead timesbetween the successive transmitted bursts. The frequency of theultrasonic transducer's transmit signal will differ from that of thereturn signal, because the scattering red blood cells within the radialartery are moving. Thus, the return signal, effectively, is frequencymodulated by the blood flow velocity.

A driving and receiving circuit generates electrical pulses which, whenapplied to the transducer, produce acoustic energy having a frequency onthe order of 8 MHz, a pulse width or duration of approximately 8microseconds, and a pulse repetition interval (PRI) of approximately 16μs, although other values of frequency, pulse width, and PRI may beused. In one embodiment, the transducer 84 emits an 8 microsecond pulse,which is followed by an 8 microsecond “listen” period, every 16microseconds. The echoes from these pulses are received by theultrasonic transducer 84 during the listen period. The ultrasonictransducer can be a ceramic piezoelectric device of the type well knownin the art, although other types may be substituted.

An analog signal representative of the Doppler frequency of the echo isreceived by the transducer and converted to a digital representation bythe ADC, and supplied to the CPU for signal processing. Within the CPU,the digitized Doppler frequency is scaled to compute the blood flowvelocity within the artery based on the Doppler frequency. Based on thereal time the blood flow velocity, the CPU applies the vital model tothe corresponding blood flow velocity to produce the estimated bloodpressure value.

Prior to operation, calibration is done using a calibration device andthe monitoring device to simultaneously collect blood pressure values(systolic, diastolic pressures) and a corresponding blood flow velocitygenerated by the monitoring device. The calibration device is attachedto the base station and measures systolic and diastolic blood pressureusing a cuff-based blood pressure monitoring device that includes amotor-controlled pump and data-processing electronics. While thecuff-based blood pressure monitoring device collects patient data, thetransducer collects patient data in parallel and through the watch'sradio transmitter, blood flow velocity is sent to the base station forgenerating a computer model that converts the blood flow velocityinformation into systolic and diastolic blood pressure values and thisinformation is sent wirelessly from the base station to the watch fordisplay and to a remote server if needed. This process is repeated at alater time (e.g., 15 minutes later) to collect a second set ofcalibration parameters. In one embodiment, the computer model fits theblood flow velocity to the systolic/diastolic values. In anotherembodiment, the computer trains a neural network or HMM to recognize thesystolic and diastolic blood pressure values.

After the computer model has been generated, the system is ready forreal-time blood pressure monitoring. In an acoustic embodiment, thetransducer directs ultrasound at the patient's artery and subsequentlylistens to the echos therefrom. The echoes are used to determine bloodflow, which is fed to the computer model to generate the systolic anddiastolic pressure values as well as heart rate value. The CPU's outputsignal is then converted to a form useful to the user such as a digitalor analog display, computer data file, or audible indicator. The outputsignal can drive a speaker to enable an operator to hear arepresentation of the Doppler signals and thereby to determine when thetransducer is located approximately over the radial artery. The outputsignal can also be wirelessly sent to a base station for subsequentanalysis by a physician, nurse, caregiver, or treating professional. Theoutput signal can also be analyzed for medical attention and medicaltreatment.

It is noted that while the above embodiment utilizes a preselected pulseduration of 8 microseconds and pulse repetition interval of 16microseconds, other acoustic sampling techniques may be used inconjunction with the invention. For example, in a second embodiment ofthe ultrasonic driver and receiver circuit (not shown), the acousticpulses are range-gated with a more complex implementation of the gatelogic. As is well known in the signal processing arts, range-gating is atechnique by which the pulse-to-pulse interval is varied based on thereceipt of range information from earlier emitted and reflected pulses.Using this technique, the system may be “tuned” to receive echoesfalling within a specific temporal window which is chosen based on therange of the echo-producing entity in relation to the acoustic source.The delay time before the gate is turned on determines the depth of thesample volume. The amount of time the gate is activated establishes theaxial length of the sample volume. Thus, as the acoustic source (in thiscase the ultrasonic transducer 84) is tuned to the echo-producing entity(red blood cells, or arterial walls), the pulse repetition interval isshortened such that the system may obtain more samples per unit time,thereby increasing its resolution. It will be recognized that otheracoustic processing techniques may also be used, all of which areconsidered to be equivalent.

In one optical embodiment, the transducer can be an optical transducer.The optical transducer can be a light source and a photo-detectorembedded in the wrist band portions 1374. The light source can belight-emitting diodes that generate red (λ^(˜)630 nm) and infrared(λ^(˜)900 nm) radiation, for example. The light source and thephoto-detector are slidably adjustable and can be moved along the wristband to optimize beam transmission and pick up. As the heart pumps bloodthrough the patient's finger, blood cells absorb and transmit varyingamounts of the red and infrared radiation depending on how much oxygenbinds to the cells' hemoglobin. The photo-detector detects transmissionat the predetermined wavelengths, for example red and infraredwavelengths, and provides the detected transmission to a pulse-oximetrycircuit embedded within the wrist-watch. The output of thepulse-oximetry circuit is digitized into a time-dependent opticalwaveform, which is then sent back to the pulse-oximetry circuit andanalyzed to determine the user's vital signs.

In the electromagnetic sensor embodiment, the wrist band 1374 is aflexible plastic material incorporated with a flexible magnet. Themagnet provides a magnetic field, and one or more electrodes similar toelectrode 1383 are positioned on the wrist band to measure voltage dropswhich are proportional to the blood velocity. The electromagneticembodiment may be mounted on the upper arm of the patient, on the ankleor on the neck where peripheral blood vessels pass through and theirblood velocity may be measured with minimal interruptions. The flexiblemagnet produces a pseudo-uniform (non-gradient) magnetic field. Themagnetic field can be normal to the blood flow direction when wrist band1374 is mounted on the user's wrist or may be a rotative pseudo-uniformmagnetic field so that the magnetic field is in a transversal directionin respect to the blood flow direction. The electrode output signals areprocessed to obtain a differential measurement enhancing the signal tonoise ratio. The flow information is derived based on the periodicity ofthe signals. The decoded signal is filtered over several periods andthen analyzed for changes used to estimate artery and vein blood flow.Systemic stroke volume and cardiac output may be calculated from theperipheral SV index value.

The wrist-band 1374 further contains an antenna 1376 for transmitting orreceiving radio frequency signals. The wristband 1374 and the antenna1376 inside the band are mechanically coupled to the top and bottomsides of the wrist-watch housing 1380. Further, the antenna 1376 iselectrically coupled to a radio frequency transmitter and receiver forwireless communications with another computer or another user. Althougha wrist-band is disclosed, a number of substitutes may be used,including a belt, a ring holder, a brace, or a bracelet, among othersuitable substitutes known to one skilled in the art. The housing 1380contains the processor and associated peripherals to provide thehuman-machine interface. A display 1382 is located on the front sectionof the housing 1380. A speaker 1384, a microphone 1388, and a pluralityof push-button switches 1386 and 1390 are also located on the frontsection of housing 1380.

The electronic circuitry housed in the watch case 1380 detects adverseconditions such as falls or seizures. In one implementation, thecircuitry can recognize speech, namely utterances of spoken words by theuser, and converting the utterances into digital signals. The circuitryfor detecting and processing speech to be sent from the wristwatch tothe base station 20 over the mesh network includes a central processingunit (CPU) connected to a ROM/RAM memory via a bus. The CPU is apreferably low power 16-bit or 32-bit microprocessor and the memory ispreferably a high density, low-power RAM. The CPU is coupled via the busto processor wake-up logic, one or more accelerometers to detect suddenmovement in a patient, an ADC 102 which receives speech input from themicrophone. The ADC converts the analog signal produced by themicrophone into a sequence of digital values representing the amplitudeof the signal produced by the microphone at a sequence of evenly spacedtimes. The CPU is also coupled to a digital to analog (D/A) converter,which drives the speaker to communicate with the user. Speech signalsfrom the microphone are first amplified, pass through an antialiasingfilter before being sampled. The front-end processing includes anamplifier, a bandpass filter to avoid antialiasing, and ananalog-to-digital (A/D) converter or a CODEC. To minimize space, theADC, the DAC and the interface for wireless transceiver and switches maybe integrated into one integrated circuit to save space. In oneembodiment, the wrist watch acts as a walkie-talkie so that voice isreceived over the mesh network by the base station 20 and then deliveredto a call center over the POTS or PSTN network. In another embodiment,voice is provided to the call center using the Internet through suitableVOIP techniques. In one embodiment, speech recognition such as a speechrecognizer is discussed in U.S. Pat. No. 6,070,140 by the inventor ofthe instant invention, the content of which is incorporated byreference.

FIG. 6B shows an exemplary mesh network working with the wearableappliance of FIG. 6A. Data collected and communicated on the display1382 of the watch as well as voice is transmitted to a base station 1390for communicating over a network to an authorized party 1394. The watchand the base station is part of a mesh network that may communicate witha medicine cabinet to detect opening or to each medicine container 1391to detect medication compliance. Other devices include mesh networkthermometers, scales, or exercise devices. The mesh network alsoincludes a plurality of home/room appliances 1392-1399. The ability totransmit voice is useful in the case the patient has fallen down andcannot walk to the base station 1390 to request help. Hence, in oneembodiment, the watch captures voice from the user and transmits thevoice over the Zigbee mesh network to the base station 1390. The basestation 1390 in turn dials out to an authorized third party to allowvoice communication and at the same time transmits the collected patientvital parameter data and identifying information so that help can bedispatched quickly, efficiently and error-free. In one embodiment, thebase station 1390 is a POTS telephone base station connected to thewired phone network. In a second embodiment, the base station 1390 canbe a cellular telephone connected to a cellular network for voice anddata transmission. In a third embodiment, the base station 1390 can be aWiMAX or 802.16 standard base station that can communicate VOIP and dataover a wide area network. Alternatively, the base station cancommunicate over POTS and a wireless network such as cellular or WiMAXor both.

In one embodiment, the processor and transceiver on the watch and thebase station conform to the Zigbee protocol. ZigBee is a cost-effective,standards-based wireless networking solution that supports lowdata-rates, low-power consumption, security, and reliability. Singlechip Zigbee controllers with wireless transceivers built-in include theChipcon/Ember CC2420: Single-chip 802.15.4 radio transceiver and theFreeScale single chip Zigbee and microcontroller. In variousembodiments, the processor communicates with a Z axis accelerometermeasures the patient's up and down motion and/or an X and Y axisaccelerometer measures the patient's forward and side movements. In oneembodiment, EKG and/or blood pressure parameters can be captured by theprocessor. The controllers upload the captured data when the memory isfull or while in wireless contact with other Zigbee nodes.

The wristwatch device can also be used to control home automation. Theuser can have flexible management of lighting, heating and coolingsystems from anywhere in the home. The watch automates control ofmultiple home systems to improve conservation, convenience and safety.The watch can capture highly detailed electric, water and gas utilityusage data and embed intelligence to optimize consumption of naturalresources. The system is convenient in that it can be installed,upgraded and networked without wires. The patient can receive automaticnotification upon detection of unusual events in his or her home. Forexample, if smoke or carbon monoxide detectors detect a problem, thewrist-watch can buzz or vibrate to alert the user and the central hubtriggers selected lights to illuminate the safest exit route.

In another embodiment, the watch serves a key fob allowing the user towirelessly unlock doors controlled by Zigbee wireless receiver. In thisembodiment, when the user is within range, the door Zigbee transceiverreceives a request to unlock the door, and the Zigbee transceiver on thedoor transmits an authentication request using suitable securitymechanism. Upon entry, the Zigbee doorlock device sends access signalsto the lighting, air-conditioning and entertainment systems, amongothers. The lights and temperature are automatically set topre-programmed preferences when the user's presence is detected.

Although Zigbee is mentioned as an exemplary protocol, other protocolssuch as UWB, Bluetooth, WiFi and WiMAX can be used as well.

While the foregoing addresses the needs of the elderly, the system canassist infants as well. Much attention has been given to ways to reducea risk of dying from Sudden Infant Death Syndrome (SIDS), an afflictionwhich threatens infants who have died in their sleep for heretoforeunknown reasons. Many different explanations for this syndrome and waysto prevent the syndrome are found in the literature. It is thought thatinfants which sleep on their backs may be at risk of death because ofthe danger of formula regurgitation and liquid aspiration into thelungs. It has been thought that infants of six (6) months or less do nothave the motor skills or body muscular development to regulate movementsresponsive to correcting breathing problems that may occur during sleep.

In an exemplary system to detect and minimize SIDS problem in an infantpatient, a diaper pad is used to hold an array of integrated sensors andthe pad can be placed over a diaper, clothing, or blanket. Theintegrated sensors can provide data for measuring position, temperature,sound, vibration, movement, and optionally other physical propertiesthrough additional sensors. Each pad can have sensors that provide oneor more of the above data. The sensors can be added or removed asnecessary depending on the type of data being collected.

The sensor should be water proof and disposable. The sensor can beswitch on/off locally or remotely. The sensor can be removable or clipon easily. The sensor can store or beam out information for analysispurpose, e.g. store body temperature every 5 seconds. The sensor can beturn-on for other purposed, e.g. diaper wet, it will beep and allow ababy care provider to take care of the business in time. The array ofsensors can be self selective, e.g., when one sensor can detect strongheart beat, it will turn off others to do so.

The sensor can be used for drug delivery system, e.g. when patient hasabdomen pain, soothing drug can be applied, based on the level of painthe sensor detects, different dose of drugs will be applied.

The array of sensors may allow the selection and analysis of zones ofsensors in the areas of interest such as the abdomen area. Each sensorarray has a low spatial resolution: approximately 10 cm between eachsensor. In addition to lower cost due to the low number of sensors, itis also possible to modify the data collection rate from certain sensorsthat are providing high-quality data. Other sensors may include thoseworn on the body, such as in watch bands, finger rings, or adhesivesensors, but telemetry, not wires, would be used to communicate with thecontroller.

The sensor can be passive device such as a reader, which mounted nearthe crib can active it from time to time. In any emergency situation,the sensor automatically signals a different state which the reader candetect.

The sensor can be active and powered by body motion or body heat. Thesensor can detect low battery situation and warn the user to provide areplacement battery.

In one embodiment, a plurality of sensors attached to the infantcollects the vital parameters. For example, the sensors can be attachedto the infant's clothing (shirt or pant), diaper, undergarment or bedsheet, bed linen, or bed spread.

The patient may wear one or more sensors, for example devices forsensing ECG, EKG, blood pressure, sugar level, weight, temperature andpressure, among others. In one embodiment, an optical temperature sensorcan be used. In another embodiment, a temperature thermistor can be usedto sense patient temperature. In another embodiment, a fat scale sensorcan be used to detect the patient's fat content. In yet anotherembodiment, a pressure sensor such as a MEMS sensor can be used to sensepressure on the patient.

In one embodiment, the sensors are mounted on the patient's wrist (suchas a wristwatch sensor) and other convenient anatomical locations.Exemplary sensors include standard medical diagnostics for detecting thebody's electrical signals emanating from muscles (EMG and EOG) and brain(EEG) and cardiovascular system (ECG). Leg sensors can includepiezoelectric accelerometers designed to give qualitative assessment oflimb movement. Additionally, thoracic and abdominal bands used tomeasure expansion and contraction of the thorax and abdomenrespectively. A small sensor can be mounted on the subject's finger inorder to detect blood-oxygen levels and pulse rate. Additionally, amicrophone can be attached to throat and used in sleep diagnosticrecordings for detecting breathing and other noise. One or more positionsensors can be used for detecting orientation of body (lying on leftside, right side or back) during sleep diagnostic recordings. Each ofsensors can individually transmit data to the server 20 using wired orwireless transmission. Alternatively, all sensors can be fed through acommon bus into a single transceiver for wired or wireless transmission.The transmission can be done using a magnetic medium such as a floppydisk or a flash memory card, or can be done using infrared or radionetwork link, among others.

In one embodiment, the sensors for monitoring vital signs are enclosedin a wrist-watch sized case supported on a wrist band. The sensors canbe attached to the back of the case. For example, in one embodiment,Cygnus' AutoSensor (Redwood City, Calif.) is used as a glucose sensor. Alow electric current pulls glucose through the skin. Glucose isaccumulated in two gel collection discs in the AutoSensor. TheAutoSensor measures the glucose and a reading is displayed by the watch.

In another embodiment, EKG/ECG contact points are positioned on the backof the wrist-watch case. In yet another embodiment that providescontinuous, beat-to-beat wrist arterial pulse rate measurements, apressure sensor is housed in a casing with a ‘free-floating’ plunger asthe sensor applanates the radial artery. A strap provides a constantforce for effective applanation and ensuring the position of the sensorhousing to remain constant after any wrist movements. The change in theelectrical signals due to change in pressure is detected as a result ofthe piezoresistive nature of the sensor are then analyzed to arrive atvarious arterial pressure, systolic pressure, diastolic pressure, timeindices, and other blood pressure parameters.

The heartbeat detector can be one of: EKG detector, ECG detector,optical detector, ultrasonic detector, or microphone/digital stethoscopefor picking up heart sound. In one embodiment, one EKG/ECG contact pointis provided on the back of the wrist watch case and one or more EKG/ECGcontact points are provided on the surface of the watch so that when auser's finger or skin touches the contact points, an electrical signalindicative of heartbeat activity is generated. An electrocardiogram(ECG) or EKG is a graphic tracing of the voltage generated by thecardiac or heart muscle during a heartbeat. It provides very accurateevaluation of the performance of the heart. The heart generates anelectrochemical impulse that spreads out in the heart in such a fashionas to cause the cells to contract and relax in a timely order and thusgive the heart a pumping characteristic. This sequence is initiated by agroup of nerve cells called the sinoatrial (SA) node resulting in apolarization and depolarization of the cells of the heart. Because thisaction is electrical in nature and because the body is conductive withits fluid content, this electrochemical action can be measured at thesurface of the body. An actual voltage potential of approximately 1 mVdevelops between various body points. This can be measured by placingelectrode contacts on the body. The four extremities and the chest wallhave become standard sites for applying the electrodes. Standardizingelectrocardiograms makes it possible to compare them as taken fromperson to person and from time to time from the same person. The normalelectrocardiogram shows typical upward and downward deflections thatreflect the alternate contraction of the atria (the two upper chambers)and of the ventricles (the two lower chambers) of the heart. Thevoltages produced represent pressures exerted by the heart muscles inone pumping cycle. The first upward deflection, P, is due to atriacontraction and is known as the atrial complex. The other deflections,Q, R, S, and T, are all due to the action of the ventricles and areknown as the ventricular complexes. Any deviation from the norm in aparticular electrocardiogram is indicative of a possible heart disorder.

The CPU measures the time duration between the sequential pulses andconverts each such measurement into a corresponding timing measurementindicative of heart rate. The CPU also processes a predetermined numberof most recently occurring timing measurements in a prescribed fashion,to produce an estimate of heartbeat rate for display on a display deviceon the watch and/or for transmission over the wireless network. Thisestimate is updated with the occurrence of each successive pulse.

In one embodiment, the CPU produces the estimate of heartbeat rate byfirst averaging a plurality of measurements, then adjusting theparticular one of the measurements that differs most from the average tobe equal to that average, and finally computing an adjusted averagebased on the adjusted set of measurements. The process may repeat theforegoing operations a number of times so that the estimate of heartbeatrate is substantially unaffected by the occurrence of heartbeatartifacts.

In one EKG or ECG detector, the heartbeat detection circuitry includes adifferential amplifier for amplifying the signal transmitted from theEKG/ECG electrodes and for converting it into single-ended form, and abandpass filter and a 60 Hz notch filter for removing background noise.The CPU measures the time durations between the successive pulses andestimates the heartbeat rate. The time durations between the successivepulses of the pulse sequence signal provides an estimate of heartbeatrate. Each time duration measurement is first converted to acorresponding rate, preferably expressed in beats per minute (bpm), andthen stored in a file, taking the place of the earliest measurementpreviously stored. After a new measurement is entered into the file, thestored measurements are averaged, to produce an average ratemeasurement. The CPU optionally determines which of the storedmeasurements differs most from the average, and replaces thatmeasurement with the average.

Upon initiation, the CPU increments a period timer used in measuring thetime duration between successive pulses. This timer is incremented insteps of about two milliseconds in one embodiment. It is then determinedwhether or not a pulse has occurred during the previous twomilliseconds. If it has not, the CPU returns to the initial step ofincrementing the period timer. If a heartbeat has occurred, on the otherhand, the CPU converts the time duration measurement currently stored inthe period timer to a corresponding heartbeat rate, preferably expressedin bpm. After the heartbeat rate measurement is computed, the CPUdetermines whether or not the computed rate is intermediate prescribedthresholds of 20 bpm and 240 bpm. If it is not, it is assumed that thedetected pulse was not in fact a heartbeat and the period timer iscleared.

In an optical heartbeat detector embodiment, an optical transducer ispositioned on a finger, wrist, or ear lobe. The ear, wrist or fingerpulse oximeter waveform is then analyzed to extract the beat-to-beatamplitude, area, and width (half height) measurements. The oximeterwaveform is used to generate heartbeat rate in this embodiment. In oneimplementation, a reflective sensor such as the Honeywell HLC1395 can beused. The device emits lights from a window in the infrared spectrum andreceives reflected light in a second window. When the heart beats, bloodflow increases temporarily and more red blood cells flow through thewindows, which increases the light reflected back to the detector. Thelight can be reflected, refracted, scattered, and absorbed by one ormore detectors. Suitable noise reduction is done, and the resultingoptical waveform is captured by the CPU.

In another optical embodiment, blood pressure is estimated from theoptical reading using a mathematical model such as a linear correlationwith a known blood pressure reading. In this embodiment, the pulseoximeter readings are compared to the blood-pressure readings from aknown working blood pressure measurement device during calibration.Using these measurements the linear equation is developed relatingoximeter output waveform such as width to blood-pressure (systolic, meanand pulse pressure). In one embodiment, a transform (such as a Fourieranalysis or a Wavelet transform) of the oximeter output can be used togenerate a model to relate the oximeter output waveform to the bloodpressure. Other non-linear math model or relationship can be determinedto relate the oximeter waveform to the blood pressure.

In one implementation, the pulse oximeter probe and a blood pressurecuff are placed on the corresponding contralateral limb to theoscillometric (Dinamap 8100; Critikon, Inc, Tampa, Fla., USA) cuff site.The pulse oximeter captures data on plethysmographic waveform, heartrate, and oxygen saturation. Simultaneous blood pressure measurementswere obtained from the oscillometric device, and the pulse oximeter.Systolic, diastolic, and mean blood pressures are recorded from theoscillometric device. This information is used derive calibrationparameters relating the pulse oximeter output to the expected bloodpressure. During real time operation, the calibration parameters areapplied to the oximeter output to predict blood pressure in a continuousor in a periodic fashion. In yet another embodiment, the device includesan accelerometer or alternative motion-detecting device to determinewhen the patient' hand is at rest, thereby reducing motion-relatedartifacts introduced to the measurement during calibration and/oroperation. The accelerometer can also function as a falls detectiondevice.

In an ultrasonic embodiment, a piezo film sensor element is placed onthe wristwatch band. The sensor can be the SDT1-028K made by MeasurementSpecialties, Inc. The sensor should have features such as: (a) it issensitive to low level mechanical movements, (b) it has an electrostaticshield located on both sides of the element (to minimize 50/60 Hz ACline interference), (c) it is responsive to low frequency movements inthe 0.7-12 Hz range of interest. A filter/amplifier circuit has athree-pole low pass filter with a lower (−3 dB) cutoff frequency atabout 12-13 Hz. The low-pass filter prevents unwanted 50/60 Hz AC lineinterference from entering the sensor. However, the piezo film elementhas a wide band frequency response so the filter also attenuates anyextraneous sound waves or vibrations that get into the piezo element.The DC gain is about +30 dB.

Waveform averaging can be used to reduce noise. It reinforces thewaveform of interest by minimizing the effect of any random noise. Thesepulses were obtained when the arm was motionless. If the arm was movedwhile capturing the data the waveform did not look nearly as clean.That's because motion of the arm causes the sonic vibrations to enterthe piezo film through the arm or by way of the cable. An accelerometeris used to detect arm movement and used to remove inappropriate datacapture.

In one embodiment that can determine blood pressure, two piezo filmsensors and filter/amplifier circuits can be configured as anon-invasive velocity type blood pressure monitor. One sensor can be onthe wrist and the other can be located on the inner left elbow at thesame location where Korotkoff sounds are monitored during traditionalblood pressure measurements with a spygmometer. The correlation betweenpulse delay and blood pressure is well known in the art of non-invasiveblood pressure monitors.

In yet another embodiment, an ultrasonic transducer generates andtransmits an acoustic wave into the user's body such as the wrist orfinger. The transducer subsequently receives pressure waves in the formof echoes resulting from the transmitted acoustic waves. In oneembodiment, an ultrasonic driving and receiving circuit generateselectrical pulses which, when applied to the transducer produce acousticenergy having a frequency on the order of 8 MHz, a pulse width orduration of approximately 8 microseconds, and a pulse repetitioninterval (PRI) of approximately 16 microseconds, although other valuesof frequency, pulse width, and PRI may be used. Hence, the transduceremits an 8 microsecond ultrasonic pulse, which is followed by an 8microsecond “listen” period, every 16 microseconds. The echoes fromthese pulses are received by the ultrasonic transducer during the listenperiod. The ultrasonic transducer can be a ceramic piezoelectric deviceof the type well known in the art, although other types may besubstituted. The transducer converts the received acoustic signal to anelectrical signal, which is then supplied to the receiving section ofthe ultrasonic driver and receiver circuit 616, which contains tworeceiver circuits. The output of the first receiver circuit is an analogsignal representative of the Doppler frequency of the echo received bythe transducer which is digitized and supplied to the CPU. Within theCPU, the digitized Doppler frequency is scaled to compute the bloodvelocity within the artery based on the Doppler frequency. Thetime-frequency distribution of the blood velocity is then computed.Finally, the CPU maps in time the peak of the time-frequencydistribution to the corresponding pressure waveform to produce theestimated mean arterial pressure (MAP). The output of the ultrasonicreceiver circuit is an analog echo signal proportional to absorption ofthe transmitted frequencies by blood or tissue. This analog signal isdigitized and process so that each group of echoes, generated for adifferent transversal position, is integrated to determine a mean value.The mean echo values are compared to determine the minimum value, whichis caused by direct positioning over the artery. In one embodiment, thedevice includes an accelerometer or alternative motion-detecting deviceto determine when the patient' hand is at rest, thereby reducingmotion-related artifacts introduced to the measurement.

In yet another ultrasonic embodiment, a transducer includes a first anda second piezoelectric crystal, wherein the crystals are positioned atan angle to each other, and wherein the angle is determined based on thedistance of the transducer to the living subject. The firstpiezoelectric crystal is energized by an original ultrasonic frequencysignal, wherein the original ultrasonic frequency signal is reflectedoff the living subject and received by the second piezoelectric crystal.More specifically, the system includes a pair of piezoelectric crystalsat an angle to each other, wherein the angle is determined by the depthof the object being monitored. If the object is the radial artery of ahuman subject (e.g., adult, infant), the angle of the two crystals withrespect to the direction of the blood flow would be about 5 to about 20degrees. One of the crystals is energized at an ultrasonic frequency.The signal is then reflected back by the user's wrist and picked up bythe second crystal. The frequency received is either higher or lowerthan the original frequency depending upon the direction and the speedof the fluidic mass flow. For example, when blood flow is monitored, thedirection of flow is fixed. Thus, the Doppler frequency which is thedifference between the original and the reflected frequency depends onlyupon the speed of the blood flow. Ultrasonic energy is delivered to oneof the two piezoelectric elements in the module by the power amplifier.The other element picks up the reflected ultrasonic signal as Dopplerfrequencies.

In a digital stethoscope embodiment, a microphone or a piezoelectrictransducer is placed near the wrist artery to pick up heart rateinformation. In one embodiment, the microphone sensor and optionally theEKG sensor are place on the wrist band 1374 of the watch to analyze theacoustic signal or signals emanating from the cardiovascular system and,optionally can combine the sound with an electric signal (EKG) emanatingfrom the cardiovascular system and/or an acoustic signal emanating fromthe respiratory system. The system can perform automated auscultation ofthe cardiovascular system, the respiratory system, or both. For example,the system can differentiate pathological from benign heart murmurs,detect cardiovascular diseases or conditions that might otherwise escapeattention, recommend that the patient go through for a diagnostic studysuch as an echocardiography or to a specialist, monitor the course of adisease and the effects of therapy, decide when additional therapy orintervention is necessary, and providing a more objective basis for thedecision(s) made. In one embodiment, the analysis includes selecting oneor more beats for analysis, wherein each beat comprises an acousticsignal emanating from the cardiovascular system; performing atime-frequency analysis of beats selected for analysis so as to provideinformation regarding the distribution of energy, the relativedistribution of energy, or both, over different frequency ranges at oneor more points in the cardiac cycle; and processing the information toreach a clinically relevant conclusion or recommendation. In anotherimplementation, the system selects one or more beats for analysis,wherein each beat comprises an acoustic signal emanating from thecardiovascular system; performs a time-frequency analysis of beatsselected for analysis so as to provide information regarding thedistribution of energy, the relative distribution of energy, or both,over different frequency ranges at one or more points in the cardiaccycle; and present information derived at least in part from theacoustic signal, wherein the information comprises one or more itemsselected from the group consisting of: a visual or audio presentation ofa prototypical beat, a display of the time-frequency decomposition ofone or more beats or prototypical beats, and a playback of the acousticsignal at a reduced rate with preservation of frequency content.

In an electromagnetic embodiment where the wrist band incorporates aflexible magnet to provide a magnetic field and one or more electrodespositioned on the wrist band to measure voltage drops which areproportional to the blood velocity, instantaneously variation of theflow can be detected but not artery flow by itself. To estimate the flowof blood in the artery, the user or an actuator such as motorized cufftemporarily stops the blood flow in the vein by applying externalpressure or by any other method. During the period of time in which thevein flow is occluded, the decay of the artery flow is measured. Thismeasurement may be used for zeroing the sensor and may be used in amodel for estimating the steady artery flow. The decay in artery flowdue to occlusion of veins is measured to arrive at a model the rate ofartery decay. The system then estimates an average artery flow beforeocclusion. The blood flow can then be related to the blood pressure.

In another embodiment, an ionic flow sensor is used with a drivingelectrode that produces a pulsatile current. The pulsatile currentcauses a separation of positive and negative charges that flows in theblood of the arteries and veins passing in the wrist area. Usingelectrophoresis principle, the resistance of the volume surrounded bythe source first decreases and then increases. The difference inresistance in the blood acts as a mark that moves according to the flowof blood so that marks are flowing in opposite directions by arteriesand veins.

In the above embodiments, accelerometer information is used to detectthat the patient is at rest prior to making a blood pressure measurementand estimation. Further, a temperature sensor may be incorporated sothat the temperature is known at any minute. The processor correlatesthe temperature measurement to the blood flow measurement forcalibration purposes.

In another embodiment, the automatic identification of the first,second, third and fourth heart sounds (S1, S2, S3, S4) is done. In yetanother embodiment, based on the heart sound, the system analyzes thepatient for mitral valve prolapse. The system performs a time-frequencyanalysis of an acoustic signal emanating from the subject'scardiovascular system and examines the energy content of the signal inone or more frequency bands, particularly higher frequency bands, inorder to determine whether a subject suffers from mitral valve prolapse.

FIG. 7 shows an exemplary mesh network that includes the wrist-watch ofFIG. 6 in communication with a mesh network including a telephone suchas a wired telephone as well as a cordless telephone. In one embodiment,the mesh network is an IEEE 802.15.4 (ZigBee) network. IEEE 802.15.4defines two device types; the reduced function device (RFD) and the fullfunction device (FFD). In ZigBee these are referred to as the ZigBeePhysical Device types. In a ZigBee network a node can have three roles:ZigBee Coordinator, ZigBee Router, and ZigBee End Device. These are theZigBee Logical Device types. The main responsibility of a ZigBeeCoordinator is to establish a network and to define its main parameters(e.g. choosing a radio-frequency channel and defining a unique networkidentifier). One can extend the communication range of a network byusing ZigBee Routers. These can act as relays between devices that aretoo far apart to communicate directly. ZigBee End Devices do notparticipate in routing. An FFD can talk to RFDs or other FFDs, while anRFD can talk only to an FFD. An RFD is intended for applications thatare extremely simple, such as a light switch or a passive infraredsensor; they do not have the need to send large amounts of data and mayonly associate with a single FFD at a time. Consequently, the RFD can beimplemented using minimal resources and memory capacity and have lowercost than an FFD. An FFD can be used to implement all three ZigBeeLogical Device types, while an RFD can take the role as an End Device.

One embodiment supports a multicluster-multihop network assembly toenable communication among every node in a distribution of nodes. Thealgorithm should ensure total connectivity, given a network distributionthat will allow total connectivity. One such algorithm of an embodimentis described in U.S. Pat. No. 6,832,251, the content of which isincorporated by referenced. The '251 algorithm runs on each nodeindependently. Consequently, the algorithm does not have globalknowledge of network topology, only local knowledge of its immediateneighborhood. This makes it well suited to a wide variety ofapplications in which the topology may be time-varying, and the numberof nodes may be unknown. Initially, all nodes consider themselvesremotes on cluster zero. The assembly algorithm floods one packet(called an assembly packet) throughout the network. As the packet isflooded, each node modifies it slightly to indicate what the next nodeshould do. The assembly packet tells a node whether it is a base or aremote, and to what cluster it belongs. If a node has seen an assemblypacket before, it will ignore all further assembly packets.

The algorithm starts by selecting (manually or automatically) a startnode. For example, this could be the first node to wake up. This startnode becomes a base on cluster 1, and floods an assembly packet to allof its neighbors, telling them to be remotes on cluster 1. These remotesin turn tell all their neighbors to be bases on cluster 2. Only nodesthat have not seen an assembly packet before will respond to thisrequest, so nodes that already have decided what to be will not changetheir status. The packet continues on, oscillating back and forthbetween “become base/become remote”, and increasing the cluster numbereach time. Since the packet is flooded to all neighbors at every step,it will reach every node in the network. Because of the oscillatingnature of the “become base/become remote” instructions, no two baseswill be adjacent. The basic algorithm establishes a multi-clusternetwork with all gateways between clusters, but self-assembly time isproportional with the size of the network. Further, it includes onlysingle hop clusters. Many generalizations are possible, however. If manynodes can begin the network nucleation, all that is required toharmonize the clusters is a mechanism that recognizes precedence (e.g.,time of nucleation, size of subnetwork), so that conflicts in boundaryclusters are resolved. Multiple-hop clusters can be enabled by means ofestablishing new clusters from nodes that are N hops distant from themaster.

Having established a network in this fashion, the masters can beoptimized either based on number of neighbors, or other criteria such asminimum energy per neighbor communication. Thus, the basic algorithm isat the heart of a number of variations that lead to a scalablemulti-cluster network that establishes itself in time, and that isnearly independent of the number of nodes, with clusters arrangedaccording to any of a wide range of optimality criteria. Networksynchronism is established at the same time as the network connections,since the assembly packet(s) convey timing information outwards fromconnected nodes.

The network nodes can be mesh network appliances to provide voicecommunications, home security, door access control, lighting control,power outlet control, dimmer control, switch control, temperaturecontrol, humidity control, carbon monoxide control, fire alarm control,blind control, shade control, window control, oven control, cookingrange control, personal computer control, entertainment console control,television control, projector control, garage door control, car control,pool temperature control, water pump control, furnace control, heatercontrol, thermostat control, electricity meter monitor, water metermonitor, gas meter monitor, or remote diagnotics. The telephone can beconnected to a cellular telephone to answer calls directed at thecellular telephone. The connection can be wired or wireless usingBluetooth or ZigBee. The telephone synchronizes calendar, contact,emails, blogs, or instant messaging with the cellular telephone.Similarly, the telephone synchronizes calendar, contact, emails, blogs,or instant messaging with a personal computer. A web server cancommunicate with the Internet through the POTS to provide information toan authorized remote user who logs into the server. A wireless routersuch as 802.11 router, 802.16 router, WiFi router, WiMAX router,Bluetooth router, X10 router can be connected to the mesh network.

A mesh network appliance can be connected to a power line to communicateX10 data to and from the mesh network. X10 is a communication protocolthat allows up to 256 X10 products to talk to each other using theexisting electrical wiring in the home. Typically, the installation issimple, a transmitter plugs (or wires) in at one location in the homeand sends its control signal (on, off, dim, bright, etc.) to a receiverwhich plugs (or wires) into another location in the home. The meshnetwork appliance translates messages intended for X10 device to berelayed over the ZigBee wireless network, and then transmitted over thepower line using a ZigBee to X10 converter appliance.

An in-door positioning system links one or more mesh network appliancesto provide location information. Inside the home or office, the radiofrequency signals have negligible multipath delay spread (for timingpurposes) over short distances. Hence, radio strength can be used as abasis for determining position. Alternatively, time of arrival can beused to determine position, or a combination of radio signal strengthand time of arrival can be used. Position estimates can also be achievedin an embodiment by beamforming, a method that exchanges time-stampedraw data among the nodes. While the processing is relatively morecostly, it yields processed data with a higher signal to noise ratio(SNR) for subsequent classification decisions, and enables estimates ofangles of arrival for targets that are outside the convex hull of theparticipating sensors. Two such clusters of ZigBee nodes can thenprovide for triangulation of distant targets. Further, beamformingenables suppression of interfering sources, by placing nulls in thesynthetic beam pattern in their directions. Another use of beamformingis in self-location of nodes when the positions of only a very smallnumber of nodes or appliances are known such as those sensors nearestthe wireless stations. In one implementation where each node knows thedistances to its neighbors due to their positions, and some smallfraction of the nodes (such as those nearest a PC with GPS) of thenetwork know their true locations. As part of the network-buildingprocedure, estimates of the locations of the nodes that lie within ornear the convex hull of the nodes with known position can be quicklygenerated. To start, the shortest distance (multihop) paths aredetermined between each reference node. All nodes on this path areassigned a location that is the simple linear average of the tworeference locations, as if the path were a straight line. A node whichlies on the intersection of two such paths is assigned the average ofthe two indicated locations. All nodes that have been assigned locationsnow serve as references. The shortest paths among these new referencenodes are computed, assigning locations to all intermediate nodes asbefore, and continuing these iterations until no further nodes getassigned locations. This will not assign initial position estimates toall sensors. The remainder can be assigned locations based on pairwiseaverages of distances to the nearest four original reference nodes. Someconsistency checks on location can be made using trigonometry and onefurther reference node to determine whether or not the node likely lieswithin the convex hull of the original four reference sensors.

In two dimensions, if two nodes have known locations, and the distancesto a third node are known from the two nodes, then trigonometry can beused to precisely determine the location of the third node. Distancesfrom another node can resolve any ambiguity. Similarly, simple geometryproduces precise calculations in three dimensions given four referencenodes. But since the references may also have uncertainty, analternative procedure is to perform a series of iterations wheresuccessive trigonometric calculations result only in a delta of movementin the position of the node. This process can determine locations ofnodes outside the convex hull of the reference sensors. It is alsoamenable to averaging over the positions of all neighbors, since therewill often be more neighbors than are strictly required to determinelocation. This will reduce the effects of distance measurement errors.Alternatively, the network can solve the complete set of equations ofintersections of hyperbola as a least squares optimization problem.

In yet another embodiment, any or all of the nodes may includetransducers for acoustic, infrared (IR), and radio frequency (RF)ranging. Therefore, the nodes have heterogeneous capabilities forranging. The heterogeneous capabilities further include differentmargins of ranging error. Furthermore, the ranging system is re-used forsensing and communication functions. For example, wideband acousticfunctionality is available for use in communicating, bistatic sensing,and ranging. Such heterogeneous capability of the sensors 40 can providefor ranging functionality in addition to communications functions. Asone example, repeated use of the communications function improvesposition determination accuracy over time. Also, when the ranging andthe timing are conducted together, they can be integrated in aself-organization protocol in order to reduce energy consumption.Moreover, information from several ranging sources is capable of beingfused to provide improved accuracy and resistance to environmentalvariability. Each ranging means is exploited as a communication means,thereby providing improved robustness in the presce of noise andinterference. Those skilled in the art will realize that there are manyarchitectural possibilities, but allowing for heterogeneity from theoutset is a component in many of the architectures.

Turning now to FIGS. 8-13, various exemplary monitoring devices areshown. In FIG. 8, a ring 130 has an opening 132 for transmitting andreceiving acoustic energy to and from the sensor 84 in an acousticimplementation. In an optical implementation, a second opening (notshown) is provided to emit an optical signal from an LED, for example,and an optical detector can be located at the opening 132 to receive theoptical signal passing through the finger wearing the ring 130. Inanother implementation, the ring has an electrically movable portion 134and rigid portions 136-138 connected thereto. The electrically movableportion 134 can squeeze the finger as directed by the CPU during anapplanation sweep to determine the arterial blood pressure.

FIG. 9 shows an alternate finger cover embodiment where a finger-mountedmodule housing the photo-detector and light source. The finger mountedmodule can be used to measure information that is processed to determinethe user's blood pressure by measuring blood flow in the user's fingerand sending the information through a wireless connection to the basestation. In one implementation, the housing is made from a flexiblepolymer material.

In an embodiment to be worn on the patient's ear lobe, the monitoringdevice can be part of an earring jewelry clipped to the ear lobe. In theimplementation of FIG. 10, the monitoring device has a jewelry body 149that contains the monitoring electronics and power source. The surfaceof the body 149 is an ornamental surface such as jade, ivory, pearl,silver, or gold, among others. The body 149 has an opening 148 thattransmits energy such as optical or acoustic energy through the ear lobeto be detected by a sensor 144 mounted on a clamp portion that issecured to the body 149 at a base 147. The energy detected through thesensor 144 is communicated through an electrical connector to theelectronics in the jewelry body 149 for processing the received energyand for performing wireless communication with a base station. In FIG.2E, a bolt 145 having a stop end 146 allows the user to adjust thepressure of the clamp against the ear lobe. In other implementations, aspring biased clip is employed to retain the clip on the wearer's earlobe. A pair of members, which snap together under pressure, arecommonly used and the spring pressure employed should be strong enoughto suit different thicknesses of the ear lobe.

FIGS. 11 and 12 show two additional embodiments of the monitoringdevice. In FIG. 11, a wearable monitoring device is shown. Themonitoring device has a body 160 comprising microphone ports 162, 164and 170 arranged in a first order noise cancelling microphonearrangement. The microphones 162 and 164 are configured to optimallyreceive distant noises, while the microphone 170 is optimized forcapturing the user's speech. A touch sensitive display 166 and aplurality of keys 168 are provided to capture hand inputs. Further, aspeaker 172 is provided to generate a verbal feedback to the user.

Turning now to FIG. 12, a jewelry-sized monitoring device isillustrated. In this embodiment, a body 172 houses a microphone port 174and a speaker port 176. The body 172 is coupled to the user via thenecklace 178 so as to provide a personal, highly accessible personalcomputer. Due to space limitations, voice input/output is an importantuser interface of the jewelry-sized computer. Although a necklace isdisclosed, one skilled in the art can use a number of other substitutessuch as a belt, a brace, a ring, or a band to secure the jewelry-sizedcomputer to the user.

FIG. 13 shows an exemplary ear phone embodiment 180. The ear phone 180has an optical transmitter 182 which emits LED wavelengths that arereceived by the optical receiver 184. The blood oximetry information isgenerated and used to determine blood pulse or blood pressure.Additionally, a module 186 contains mesh network communicationelectronics, accelerometer, and physiological sensors such as EKG/ECGsensors or temperature sensors or ultrasonic sensors. In addition, aspeaker (not shown) is provided to enable voice communication over themesh network, and a microphone 188 is provided to pick up voice duringverbal communication and pick up heart sound when the user is not usingthe microphone for voice communication. The ear phone optionally has anear canal temperature sensor for sensing temperature in a human.

FIG. 14 shows an exemplary adhesive patch embodiment. The patch may beapplied to a persons skin by anyone including the person themselves oran authorized person such as a family member or physician. The adhesivepatch is shown generally at 190 having a gauze pad 194 attached to oneside of a backing 192, preferably of plastic, and wherein the pad canhave an impermeable side 194 coating with backing 192 and a module 196which contains electronics for communicating with the mesh network andfor sensing acceleration and EKG/ECG, heart sound, microphone, opticalsensor, or ultrasonic sensor in contacts with a wearer's skin. In oneembodiment, the module 196 has a skin side that may be coated with aconductive electrode lotion or gel to improve the contact. The entirepatch described above may be covered with a plastic or foil strip toretain moisture and retard evaporation by a conductive electrode lotionor gel provided improve the electrode contact. In one embodiment, anacoustic sensor (microphone or piezoelectric sensor) and an electricalsensor such as EKG sensor contact the patient with a conductive gelmaterial. The conductive gel material provides transmissioncharacteristics so as to provide an effective acoustic impedance matchto the skin in addition to providing electrical conductivity for theelectrical sensor. The acoustic transducer can be directed mounted onthe conductive gel material substantially with or without anintermediate air buffer. The entire patch is then packaged as sterile asare other over-the-counter adhesive bandages. When the patch is wornout, the module 196 may be removed and a new patch backing 192 may beused in place of the old patch. One or more patches may be applied tothe patient's body and these patches may communicate wirelessly usingthe mesh network or alternatively they may communicate through apersonal area network using the patient's body as a communicationmedium.

The term “positional measurement,” as that term is used herein, is notlimited to longitude and latitude measurements, or to metes and bounds,but includes information in any form from which geophysical positionscan be derived. These include, but are not limited to, the distance anddirection from a known benchmark, measurements of the time required forcertain signals to travel from a known source to the geophysicallocation where the signals may be electromagnetic or other forms, ormeasured in terms of phase, range, Doppler or other units.

FIG. 15A shows a system block diagram of the network-based patientmonitoring system in a hospital or nursing home setting. The system hasa patient component 215, a server component 216, and a client component217. The patient component 215 has one or more mesh network patienttransmitters 202 for transmitting data to the central station. Thecentral server comprises one or more Web servers 205, one or morewaveform servers 204 and one or more mesh network receivers 211. Theoutput of each mesh network receiver 211 is connected to at least one ofthe waveform servers 204. The waveform servers 204 and Web the servers205 are connected to the network 105. The Web servers 205 are alsoconnected to a hospital database 230. The hospital database 230 containspatient records. In the embodiment of FIG. 15A, a plurality of nursestations provide a plurality of nurse computer user interface 208. Theuser interface 208 receives data from an applet 210 that communicateswith the waveform server 204 and updates the display of the nursecomputers for treating patients.

The network client component 217 comprises a series of workstations 106connected to the network 105. Each workstation 106 runs a World Wide Web(WWW or Web) browser application 208. Each Web browser can open a pagethat includes one or more media player applets 210. The waveform servers204 use the network 105 to send a series of messages 220 to the Webservers 205. The Web servers 205 use the network 105 to communicatemessages, shown as a path 221, to the workstations 106. The media playerapplets running on the workstations 106 use the network 105 to sendmessages over a path 223 directly to the waveform servers 204.

FIG. 15B shows a variation of the system of FIG. 15A for call centermonitoring. In this embodiment, the patient appliances 202 wirelesslycommunicate to home base stations (not shown) which are connected to thePOTS or PSTN network for voice as well as data transmission. The data iscaptured by the waveform server 204 and the voice is passed through tothe call center agent computer 207 where the agent can communicate byvoice with the patient. The call center agent can forward the call to aprofessional such as a nurse or doctor or emergency service personnel ifnecessary. Hence, the system can include a patient monitoring appliancecoupled to the POTS or PSTN through the mesh network. The patientmonitoring appliance monitors drug usage and patient falls. The patientmonitoring appliance monitors patient movement. A call center can callto the telephone to provide a human response.

In one exemplary monitoring service providing system, such as anemergency service providing system, the system includes a communicationnetwork (e.g., the Public Switch Telephone Network or PSTN or POTS), awide area communication network (e.g., TCP/IP network) in call centers.The communication network receives calls destined for one of the callcenters. In this regard, each call destined for one of the call centersis preferably associated with a particular patient, a call identifier ora call identifier of a particular set of identifiers. A call identifierassociated with an incoming call may be an identifier dialed orotherwise input by the caller. For example, the call centers may belocations for receiving calls from a particular hospital or nursinghome.

To network may analyze the automatic number information (ANI) and/orautomatic location information (ALI) associated with the call. In thisregard, well known techniques exist for analyzing the ANI and ALI of anincoming call to identify the call as originating from a particularcalling device or a particular calling area. Such techniques may beemployed by the network to determine whether an incoming call originatedfrom a calling device within an area serviced by the call centers.Moreover, if an incoming call originated from such an area and if theincoming call is associated with the particular call identifier referredto above, then the network preferably routes the call to a designatedfacility.

When a call is routed to the facility, a central data manager, which maybe implemented in software, hardware, or a combination thereof,processes the call according to techniques that will be described inmore detail hereafter and routes the call, over the wide area network,to one of the call centers depending on the ANI and/or ALI associatedwith the call. In processing the call, the central data manager mayconvert the call from one communication protocol to anothercommunication protocol, such as voice over internet protocol (VoIP), forexample, in order to increase the performance and/or efficiency of thesystem. The central data manager may also gather information to help thecall centers in processing the call. There are various techniques thatmay be employed by the central data manager to enhance the performanceand/or efficiency of the system, and examples of such techniques will bedescribed in more detail hereafter.

Various benefits may be realized by utilizing a central facility tointercept or otherwise receive a call from the network and to then routethe call to one of the call centers via WAN. For example, servingmultiple call centers with a central data manager, may help to reducetotal equipment costs. In this regard, it is not generally necessary toduplicate the processing performed by the central data manager at eachof the call centers. Thus, equipment at each of the call centers may bereduced. As more call centers are added, the equipment savings enabledby implementing equipment at the central data manager instead of thecall centers generally increases. Furthermore, the system is notdependent on any telephone company's switch for controlling the mannerin which data is communicated to the call centers. In this regard, thecentral data manager may receive a call from the network and communicatethe call to the destination call centers via any desirable communicationtechnique, such as VoIP, for example. Data security is another possiblebenefit of the exemplary system 10 as the central data manager is ableto store the data for different network providers associated withnetwork on different partitions.

While the patient interface 90 (FIG. 1A) can provide information for asingle person, FIG. 15C shows an exemplary interface to monitor aplurality of persons, while FIG. 15D shows an exemplary dash-board thatprovides summary information on the status of a plurality of persons. Asshown in FIG. 1C, for professional use such as in hospitals, nursinghomes, or retirement homes, a display can track a plurality of patients.In FIG. 15C, a warning (such as sound or visual warning in the form oflight or red flashing text) can be generated to point out the particularpatient that may need help or attention. In FIG. 15D, a magnifier glasscan be dragged over a particular individual icon to expand and showdetailed vital parameters of the individual and if available, imagesfrom the camera 10 trained on the individual for real time videofeedback. The user can initiate voice communication with the user forconfirmation purposes by clicking on a button provided on the interfaceand speaking into a microphone on the professional's workstation.

In one embodiment for professional users such as hospitals and nursinghomes, a Central Monitoring Station provides alarm and vital signoversight for a plurality of patients from a single computerworkstation. FIG. 15E shows an exemplary multi-station vital parameteruser interface for a professional embodiment, while FIG. 15F shows anexemplary trending pattern display. The clinician interface uses simplepoint and click actions with a computer mouse or trackball. Theclinician can initiate or change monitoring functions from either theCentral Station or the bedside monitor. One skilled in the art willrecognize that patient data such as EKG data can be shown either by ascrolling waveform that moves along the screen display, or by a movingbar where the waveform is essentially stationary and the bar movesacross the screen.

In one embodiment, software for the professional monitoring systemprovides a login screen to enter user name and password, together withdatabase credentials. In Select Record function, the user can select aperson, based on either entered or pre-selected criteria. From herenavigate to their demographics, medical record, etc. The system can showa persons demographics, includes aliases, people involved in their care,friends and family, previous addresses, home and work locations,alternative numbers and custom fields. The system can show all dataelements of a persons medical record. These data elements are not ‘hardwired’, but may be configured in the data dictionary to suit yourparticular requirements. It is possible to create views of the recordthat filter it to show (for instance) just the medications or diagnosis,etc. Any data element can be can be designated ‘plan able’ in the datadictionary and then scheduled. A Summary Report can be done. Example ofa report displayed in simple format, selecting particular elements anddates. As many of these reports as required can be created, going acrossall data in the system based on some criteria, with a particularselection of fields and sorting, grouping and totaling criteria. Reportscan be created that can format and analyze any data stored on theserver. The system supports OLE controls and can include graphs, barcodes, etc. These can be previewed on screen, printed out or exported ina wide variety of formats. The system also maintains a directory of allorganizations the administrator wishes to record as well as your own.These locations are then used to record the location for elements of themedical record (where applicable), work addresses for people involved inthe care and for residential addresses for people in residential care.The data elements that form the medical record are not ‘hard wired’ (iepredefined) but may be customized by the users to suit current andfuture requirements.

In one embodiment, the wearable appliance can store patient data in itsdata storage device such as flash memory. The data can includeImmunizations and dates; medications (prescriptions and supplements);physician names, addresses, phone numbers, email addresses; location anddetails of advance directives; insurance company, billing address, phonenumber, policy number; emergency contacts, addresses,home/business/pager phone numbers, email addresses. The data can includecolor or black and white photo of the wearer of the device; a thumbprint, iris print of other distinguishing physical characteristic;dental records; sample ECG or Cardiac Echo Scan.; blood type; presentmedication being taken; drug interaction precautions; drug and/orallergic reaction precautions; a description of serious preexistingmedical conditions; Emergency Medical Instructions, which could include:administering of certain suggested drugs or physical treatments; callingemergency physician numbers listed;. bringing the patient to a certaintype of clinic or facility based on religious beliefs; and living willinstructions in the case of seriously ill patients; Organ Donorinstructions; Living Will instructions which could include: instructionsfor life support or termination of treatment; notification of next ofkin and/or friends including addresses and telephone numbers; ECG trace;Cardiac Echo Scan; EEG trace; diabetes test results; x-ray scans, amongothers. The wearable appliance stores the wearer's medical records andID information. In one embodiment, to start the process new/originalmedical information is organized and edited to fit into the BWD pageformat either in physicians office or by a third party with access to apatient's medical records using the base unit storage and encryptingsoftware which can be stored in a normal pc or other compatible computerdevice. The system can encrypt the records so as to be secure andconfidential and only accessible to authorized individuals withcompatible de-encrypting software. In the event the wearer is strickenwith an emergency illness a Paramedic, EMT or Emergency Room Techniciancan use a wireless interrogator to rapidly retrieve and display thestored medical records in the wearable appliance and send the medicalrecords via wireless telemetry to a remote emergency room or physiciansoffice for rapid and life saving medical intervention in a crisissituation. In a Non-emergency Situation, the personal health informationservice is also helpful as it eliminates the hassle of repeatedlyfilling out forms when changing health plans or seeing a new physician;stores vaccination records to schools or organizations without callingthe pediatrician; or enlists the doctor's or pharmacist's advice aboutmultiple medications without carrying all the bottles to a personalvisit.

In one embodiment, a plurality of body worn sensors with in-doorpositioning can be used as an Emergency Department and Urgent CareCenter Tracking System. The system tracks time from triage to MDassessment, identifies patients that have not yet been registered,records room usage, average wait time, and average length of stay. Thesystem allows user defined “activities” so that hospitals can tracktimes and assist in improving patient flow and satisfaction. The systemcan set custom alerts and send email/pager notifications to betteridentify long patient wait times and record the number of these alertoccurrences. The system can manage room usage by identifying those roomswhich are under/over utilized. The hospital administrator can set manualor automatic alerts and generate custom reports for analysis of patientflow. The system maximizes revenue by streamlining processes andimproving throughput; improves charge capture by ensuring compliancewith regulatory standards; increases accountability by collecting clear,meaningful data; enhances risk management and QA; and decreasesliability.

FIG. 16A shows ant exemplary process to continuously determine bloodpressure of a patient. The process generates a blood pressure model of apatient (2002); determines a blood flow velocity using a piezoelectrictransducer (2004); and provides the blood flow velocity to the bloodpressure model to continuously estimate blood pressure (2006).

FIG. 16B shows another exemplary process to continuously determine bloodpressure of a patient. First, during an initialization mode, amonitoring device and calibration device are attached to patient (2010).The monitoring device generates patient blood flow velocity, whileactual blood pressure is measured by a calibration device (2012). Next,the process generates a blood pressure model based on the blood flowvelocity and the actual blood pressure (2014). Once this is done, thecalibration device can be removed (2016). Next, during an operationmode, the process periodically samples blood flow velocity from themonitoring device on a real-time basis (18) and provides the blood flowvelocity as input information to the blood pressure model to estimateblood pressure (20). This process can be done in continuously orperiodically as specified by a user.

In one embodiment, to determine blood flow velocity, acoustic pulses aregenerated and transmitted into the artery using an ultrasonic transducerpositioned near a wrist artery. These pulses are reflected by variousstructures or entities within the artery (such as the artery walls, andthe red blood cells within the subject's blood), and subsequentlyreceived as frequency shifts by the ultrasonic transducer. Next, theblood flow velocity is determined. In this process, the frequencies ofthose echoes reflected by blood cells within the blood flowing in theartery differ from that of the transmitted acoustic pulses due to themotion of the blood cells. This well known “Doppler shift” in frequencyis used to calculate the blood flow velocity. In one embodiment fordetermining blood flow velocity, the Doppler frequency is used todetermine mean blood velocity. For example, U.S. Pat. No. 6,514,211, thecontent of which is incorporated by reference, discusses blood flowvelocity using a time-frequency representation.

In one implementation, the system can obtain one or more numericalcalibration curves describing the patient's vital signs such as bloodpressure. The system can then direct energy such as infrared orultrasound at the patient's artery and detecting reflections thereof todetermine blood flow velocity from the detected reflections. The systemcan numerically fit or map the blood flow velocity to one or morecalibration parameters describing a vital-sign value. The calibrationparameters can then be compared with one or more numerical calibrationcurves to determine the blood pressure.

Additionally, the system can analyze blood pressure, and heart rate, andpulse oximetry values to characterize the user's cardiac condition.These programs, for example, may provide a report that featuresstatistical analysis of these data to determine averages, data displayedin a graphical format, trends, and comparisons to doctor-recommendedvalues.

In one embodiment, feed forward artificial neural networks (NNs) areused to classify valve-related heart disorders. The heart sounds arecaptured using the microphone or piezoelectric transducer. Relevantfeatures were extracted using several signal processing tools, discretewavelet transfer, fast fourier transform, and linear prediction coding.The heart beat sounds are processed to extract the necessary featuresby: a) denoising using wavelet analysis, b) separating one beat out ofeach record c) identifying each of the first heart sound (FHS) and thesecond heart sound (SHS). Valve problems are classified according to thetime separation between the FHS and th SHS relative to cardiac cycletime, namely whether it is greater or smaller than 20% of cardiac cycletime. In one embodiment, the NN comprises 6 nodes at both ends, with onehidden layer containing 10 nodes. In another embodiment, linearpredictive code (LPC) coefficients for each event were fed to twoseparate neural networks containing hidden neurons.

In another embodiment, a normalized energy spectrum of the sound data isobtained by applying a Fast Fourier Transform. The various spectralresolutions and frequency ranges were used as inputs into the NN tooptimize these parameters to obtain the most favorable results.

In another embodiment, the heart sounds are denoised using six-stagewavelet decomposition, thresholding, and then reconstruction. Threefeature extraction techniques were used: the Decimation method, and thewavelet method. Classification of the heart diseases is done usingHidden Markov Models (HMMs).

In yet another embodiment, a wavelet transform is applied to a window oftwo periods of heart sounds. Two analyses are realized for the signalsin the window: segmentation of first and second heart sounds, and theextraction of the features. After segmentation, feature vectors areformed by using he wavelet detail coefficients at the sixthdecomposition level. The best feature elements are analyzed by usingdynamic programming.

In another embodiment, the wavelet decomposition and reconstructionmethod extract features from the heart sound recordings. An artificialneural network classification method classifies the heart sound signalsinto physiological and pathological murmurs. The heart sounds aresegmented into four parts: the first heart sound, the systolic period,the second heart sound, and the diastolic period. The following featurescan be extracted and used in the classification algorithm: a) Peakintensity, peak timing, and the duration of the first heart sound b) theduration of the second heart sound c) peak intensity of the aorticcomponent of S2(A2) and the pulmonic component of S2 (P2) , thesplitting interval and the reverse flag of A2 and P2, and the timing ofA2 d) the duration, the three largest frequency components of thesystolic signal and the shape of the envelope of systolic murmur e) theduration the three largest frequency components of the diastolic signaland the shape of the envelope of the diastolic murmur.

In one embodiment, the time intervals between the ECG R-waves aredetected using an envelope detection process. The intervals between Rand T waves are also determined The Fourier transform is applied to thesound to detect S1 and S2. To expedite processing, the system appliesFourier transform to detect S1 in the interval 0.1-0.5 R-R. The systemlooks for S2 the intervals R-T and 0.6 R-R. S2 has an aortic componentA2 and a pulmonary component P2. The interval between these twocomponents and its changes with respiration has clinical significance.A2 sound occurs before P2, and the intensity of each component dependson the closing pressure and hence A2 is louder than P2. The third heardsound S3 results from the sudden halt in the movement of the ventriclein response to filling in early diastole after the AV valves and isnormally observed in children and young adults. The fourth heart soundS4 is caused by the sudden halt of the ventricle in response to fillingin presystole due to atrial contraction.

In yet another embodiment, the S2 is identified and a normalizedsplitting interval between A2 and P2 is determined. If there is nooverlap, A2 and P2 are determined from the heart sound. When overlapexists between A2 and P2, the sound is dechirped for identification andextraction of A2 and P2 from S2. The A2-P2 splitting interval (SI) iscalculated by computing the cross-correlation function between A2 and P2and measuring the time of occurrence of its maximum amplitude. SI isthen normalized (NSI) for heart rate as follows: NSI=SI/cardiac cycletime. The duration of the cardiac cycle can be the average interval ofQRS waves of the ECG. It could also be estimated by computing the meaninterval between a series of consecutive S1 and S2 from the heart sounddata. A non linear regressive analysis maps the relationship between thenormalized NSI and PAP. A mapping process such as a curve-fittingprocedure determines the curve that provides the best fit with thepatient data. Once the mathematical relationship is determined, NSI canbe used to provide an accurate quantitative estimate of the systolic andmean PAP relatively independent of heart rate and systemic arterialpressure.

In another embodiment, the first heart sound (S1) is detected using atime-delayed neural network (TDNN). The network consists of a singlehidden layer, with time-delayed links connecting the hidden units to thetime-frequency energy coefficients of a Morlet wavelet decomposition ofthe input phonocardiogram (PCG) signal. The neural network operates on a200 msec sliding window with each time-delay hidden unit spanning 100msec of wavelet data.

In yet another embodiment, a local signal analysis is used with aclassifier to detect, characterize, and interpret sounds correspondingto symptoms important for cardiac diagnosis. The system detects aplurality of different heart conditions. Heart sounds are automaticallysegmented into a segment of a single heart beat cycle. Each segment arethen transformed using 7 level wavelet decomposition, based on Coifman4th order wavelet kernel. The resulting vectors 4096 values, are reducedto 256 element feature vectors, this simplified the neural network andreduced noise.

In another embodiment, feature vectors are formed by using the waveletdetail and approximation coefficients at the second and sixthdecomposition levels. The classification (decision making) is performedin 4 steps: segmentation of the first and second heart sounds,normalization process, feature extraction, and classification by theartificial neural network.

In another embodiment using decision trees, the system distinguishes (1)the Aortic Stenosis (AS) from the Mitral Regurgitation (MR) and (2) theOpening Snap (OS), the Second Heart Sound Split (A2_P2) and the ThirdHeart Sound (S3). The heart sound signals are processed to detect thefirst and second heart sounds in the following steps: a) waveletdecomposition, b) calculation of normalized average Shannon Energy, c) amorphological transform action that amplifies the sharp peaks andattenuates the broad ones d) a method that selects and recovers thepeaks corresponding to S1 and S2 and rejects others e) algorithm thatdetermines the boundaries of 51 and S2 in each heart cycle f) a methodthat distinguishes S1 from S2.

In one embodiment, once the heart sound signal has been digitized andcaptured into the memory, the digitized heart sound signal isparameterized into acoustic features by a feature extractor. The outputof the feature extractor is delivered to a sound recognizer The featureextractor can include the short time energy, the zero crossing rates,the level crossing rates, the filter-bank spectrum, the linearpredictive coding (LPC), and the fractal method of analysis. Inaddition, vector quantization may be utilized in combination with anyrepresentation techniques. Further, one skilled in the art may use anauditory signal-processing model in place of the spectral models toenhance the system's robustness to noise and reverberation.

In one embodiment of the feature extractor, the digitized heart soundsignal series s(n) is put through a low-order filter, typically afirst-order finite impulse response filter, to spectrally flatten thesignal and to make the signal less susceptible to finite precisioneffects encountered later in the signal processing. The signal ispre-emphasized preferably using a fixed pre-emphasis network, orpreemphasizer. The signal can also be passed through a slowly adaptivepre-emphasizer. The preemphasized heart sound signal is next presentedto a frame blocker to be blocked into frames of N samples with adjacentframes being separated by M samples. In one implementation, frame 1contains the first 400 samples. The frame 2 also contains 400 samples,but begins at the 300th sample and continues until the 700th sample.Because the adjacent frames overlap, the resulting LPC spectral analysiswill be correlated from frame to frame. Each frame is windowed tominimize signal discontinuities at the beginning and end of each frame.The windower tapers the signal to zero at the beginning and end of eachframe. Preferably, the window used for the autocorrelation method of LPCis the Hamming window. A noise canceller operates in conjunction withthe autocorrelator to minimize noise. Noise in the heart sound patternis estimated during quiet periods, and the temporally stationary noisesources are damped by means of spectral subtraction, where theautocorrelation of a clean heart sound signal is obtained by subtractingthe autocorrelation of noise from that of corrupted heart sound. In thenoise cancellation unit, if the energy of the current frame exceeds areference threshold level, the heart is generating sound and theautocorrelation of coefficients representing noise is not updated.However, if the energy of the current frame is below the referencethreshold level, the effect of noise on the correlation coefficients issubtracted off in the spectral domain. The result is half-wave rectifiedwith proper threshold setting and then converted to the desiredautocorrelation coefficients. The output of the autocorrelator and thenoise canceller are presented to one or more parameterization units,including an LPC parameter unit, an FFT parameter unit, an auditorymodel parameter unit, a fractal parameter unit, or a wavelet parameterunit, among others. The LPC parameter is then converted into cepstralcoefficients. The cepstral coefficients are the coefficients of theFourier transform representation of the log magnitude spectrum. A filterbank spectral analysis, which uses the short-time Fourier transformation(STFT) may also be used alone or in conjunction with other parameterblocks. FFT is well known in the art of digital signal processing. Sucha transform converts a time domain signal, measured as amplitude overtime, into a frequency domain spectrum, which expresses the frequencycontent of the time domain signal as a number of different frequencybands. The FFT thus produces a vector of values corresponding to theenergy amplitude in each of the frequency bands. The FFT converts theenergy amplitude values into a logarithmic value which reducessubsequent computation since the logarithmic values are more simple toperform calculations on than the longer linear energy amplitude valuesproduced by the FFT, while representing the same dynamic range. Ways forimproving logarithmic conversions are well known in the art, one of thesimplest being use of a look-up table. In addition, the FFT modifies itsoutput to simplify computations based on the amplitude of a given frame.This modification is made by deriving an average value of the logarithmsof the amplitudes for all bands. This average value is then subtractedfrom each of a predetermined group of logarithms, representative of apredetermined group of frequencies. The predetermined group consists ofthe logarithmic values, representing each of the frequency bands. Thus,utterances are converted from acoustic data to a sequence of vectors ofk dimensions, each sequence of vectors identified as an acoustic frame,each frame represents a portion of the utterance. Alternatively,auditory modeling parameter unit can be used alone or in conjunctionwith others to improve the parameterization of heart sound signals innoisy and reverberant environments. In this approach, the filteringsection may be represented by a plurality of filters equally spaced on alog-frequency scale from 0 Hz to about 3000 Hz and having a prescribedresponse corresponding to the cochlea. The nerve fiber firing mechanismis simulated by a multilevel crossing detector at the output of eachcochlear filter. The ensemble of the multilevel crossing intervalscorresponding to the firing activity at the auditory nerve fiber-array.The interval between each successive pair of same direction, eitherpositive or negative going, crossings of each predetermined soundintensity level is determined and a count of the inverse of theseinterspike intervals of the multilevel detectors for each spectralportion is stored as a function of frequency. The resulting histogram ofthe ensemble of inverse interspike intervals forms a spectral patternthat is representative of the spectral distribution of the auditoryneural response to the input sound and is relatively insensitive tonoise The use of a plurality of logarithmically related sound intensitylevels accounts for the intensity of the input signal in a particularfrequency range. Thus, a signal of a particular frequency having highintensity peaks results in a much larger count for that frequency than alow intensity signal of the same frequency. The multiple levelhistograms of the type described herein readily indicate the intensitylevels of the nerve firing spectral distribution and cancel noiseeffects in the individual intensity level histograms. Alternatively, thefractal parameter block can further be used alone or in conjunction withothers to represent spectral information. Fractals have the property ofself similarity as the spatial scale is changed over many orders ofmagnitude. A fractal function includes both the basic form inherent in ashape and the statistical or random properties of the replacement ofthat shape in space. As is known in the art, a fractal generator employsmathematical operations known as local affine transformations. Thesetransformations are employed in the process of encoding digital datarepresenting spectral data. The encoded output constitutes a “fractaltransform” of the spectral data and consists of coefficients of theaffine transformations. Different fractal transforms correspond todifferent images or sounds.

Alternatively, a wavelet parameterization block can be used alone or inconjunction with others to generate the parameters. Like the FFT, thediscrete wavelet transform (DWT) can be viewed as a rotation in functionspace, from the input space, or time domain, to a different domain. TheDWT consists of applying a wavelet coefficient matrix hierarchically,first to the full data vector of length N, then to a smooth vector oflength N/2, then to the smooth-smooth vector of length N/4, and so on.Most of the usefulness of wavelets rests on the fact that wavelettransforms can usefully be severely truncated, or turned into sparseexpansions. In the DWT parameterization block, the wavelet transform ofthe heart sound signal is performed. The wavelet coefficients areallocated in a non-uniform, optimized manner. In general, large waveletcoefficients are quantized accurately, while small coefficients arequantized coarsely or even truncated completely to achieve theparameterization. Due to the sensitivity of the low-order cepstralcoefficients to the overall spectral slope and the sensitivity of thehigh-order cepstral coefficients to noise variations, the parametersgenerated may be weighted by a parameter weighing block, which is atapered window, so as to minimize these sensitivities. Next, a temporalderivator measures the dynamic changes in the spectra. Power featuresare also generated to enable the system to distinguish heart sound fromsilence.

After the feature extraction has been performed, the heart soundparameters are next assembled into a multidimensional vector and a largecollection of such feature signal vectors can be used to generate a muchsmaller set of vector quantized (VQ) feature signals by a vectorquantizer that cover the range of the larger collection. In addition toreducing the storage space, the VQ representation simplifies thecomputation for determining the similarity of spectral analysis vectorsand reduces the similarity computation to a look-up table ofsimilarities between pairs of codebook vectors. To reduce thequantization error and to increase the dynamic range and the precisionof the vector quantizer, the preferred embodiment partitions the featureparameters into separate codebooks, preferably three. In the preferredembodiment, the first, second and third codebooks correspond to thecepstral coefficients, the differenced cepstral coefficients, and thedifferenced power coefficients.

With conventional vector quantization, an input vector is represented bythe codeword closest to the input vector in terms of distortion. Inconventional set theory, an object either belongs to or does not belongto a set. This is in contrast to fuzzy sets where the membership of anobject to a set is not so clearly defined so that the object can be apart member of a set. Data are assigned to fuzzy sets based upon thedegree of membership therein, which ranges from 0 (no membership) to 1.0(full membership). A fuzzy set theory uses membership functions todetermine the fuzzy set or sets to which a particular data value belongsand its degree of membership therein.

To handle the variance of heart sound patterns of individuals over timeand to perform speaker adaptation in an automatic, self-organizingmanner, an adaptive clustering technique called hierarchical spectralclustering is used. Such speaker changes can result from temporary orpermanent changes in vocal tract characteristics or from environmentaleffects. Thus, the codebook performance is improved by collecting heartsound patterns over a long period of time to account for naturalvariations in speaker behavior. In one embodiment, data from the vectorquantizer is presented to one or more recognition models, including anHMM model, a dynamic time warping model, a neural network, a fuzzylogic, or a template matcher, among others. These models may be usedsingly or in combination.

In dynamic processing, at the time of recognition, dynamic programmingslides, or expands and contracts, an operating region, or window,relative to the frames of heart sound so as to align those frames withthe node models of each S1-S4 pattern to find a relatively optimal timealignment between those frames and those nodes. The dynamic processingin effect calculates the probability that a given sequence of framesmatches a given word model as a function of how well each such framematches the node model with which it has been time-aligned. The wordmodel which has the highest probability score is selected ascorresponding to the heart sound.

Dynamic programming obtains a relatively optimal time alignment betweenthe heart sound to be recognized and the nodes of each word model, whichcompensates for the unavoidable differences in speaking rates whichoccur in different utterances of the same word. In addition, sincedynamic programming scores words as a function of the fit between wordmodels and the heart sound over many frames, it usually gives thecorrect word the best score, even if the word has been slightlymisspoken or obscured by background sound. This is important, becausehumans often mispronounce words either by deleting or mispronouncingproper sounds, or by inserting sounds which do not belong.

In dynamic time warping (DTW), the input heart sound A, defined as thesampled time values A=a(1) . . . a(n), and the vocabulary candidate B,defined as the sampled time values B=b(1) . . . b(n), are matched up tominimize the discrepancy in each matched pair of samples. Computing thewarping function can be viewed as the process of finding the minimumcost path from the beginning to the end of the words, where the cost isa function of the discrepancy between the corresponding points of thetwo words to be compared. Dynamic programming considers all possiblepoints within the permitted domain for each value of i. Because the bestpath from the current point to the next point is independent of whathappens beyond that point. Thus, the total cost of [i(k), j(k)] is thecost of the point itself plus the cost of the minimum path to it.Preferably, the values of the predecessors can be kept in an M×N array,and the accumulated cost kept in a 2.times.N array to contain theaccumulated costs of the immediately preceding column and the currentcolumn. However, this method requires significant computing resources.For the heart sound recognizer to find the optimal time alignmentbetween a sequence of frames and a sequence of node models, it mustcompare most frames against a plurality of node models. One method ofreducing the amount of computation required for dynamic programming isto use pruning Pruning terminates the dynamic programming of a givenportion of heart sound against a given word model if the partialprobability score for that comparison drops below a given threshold.This greatly reduces computation, since the dynamic programming of agiven portion of heart sound against most words produces poor dynamicprogramming scores rather quickly, enabling most words to be prunedafter only a small percent of their comparison has been performed. Toreduce the computations involved, one embodiment limits the search tothat within a legal path of the warping.

A Hidden Markov model can be used in one embodiment to evaluate theprobability of occurrence of a sequence of observations O(1), O(2), . .. O(t), . . . , O(T), where each observation O(t) may be either adiscrete symbol under the VQ approach or a continuous vector. Thesequence of observations may be modeled as a probabilistic function ofan underlying Markov chain having state transitions that are notdirectly observable. The transitions between states are represented by atransition matrix A=[a(i,j)]. Each a(i,j) term of the transition matrixis the probability of making a transition to state j given that themodel is in state i. The output symbol probability of the model isrepresented by a set of functions B=[b(j)(O(t)], where the b(j)(O(t)term of the output symbol matrix is the probability of outputtingobservation 0(t), given that the model is in state j. The first state isalways constrained to be the initial state for the first time frame ofthe utterance, as only a prescribed set of left-to-right statetransitions are possible. A predetermined final state is defined fromwhich transitions to other states cannot occur. Transitions arerestricted to reentry of a state or entry to one of the next two states.Such transitions are defined in the model as transition probabilities.For example, a heart sound pattern currently having a frame of featuresignals in state 2 has a probability of reentering state 2 of a(2,2), aprobability a(2,3) of entering state 3 and a probability ofa(2,4)=1−a(2, 1)−a(2,2) of entering state 4. The probability a(2, 1) ofentering state 1 or the probability a(2,5) of entering state 5 is zeroand the sum of the probabilities a(2,1) through a(2,5) is one. Althoughthe preferred embodiment restricts the flow graphs to the present stateor to the next two states, one skilled in the art can build an HMM modelwithout any transition restrictions.

The Markov model is formed for a reference pattern from a plurality ofsequences of training patterns and the output symbol probabilities aremultivariate Gaussian function probability densities. The heart soundtraverses through the feature extractor. During learning, the resultingfeature vector series is processed by a parameter estimator, whoseoutput is provided to the hidden Markov model. The hidden Markov modelis used to derive a set of reference pattern templates, each templaterepresentative of an identified S1-S4 pattern in a vocabulary set ofreference patterns. The Markov model reference templates are nextutilized to classify a sequence of observations into one of thereference patterns based on the probability of generating theobservations from each Markov model reference pattern template. Duringrecognition, the unknown pattern can then be identified as the referencepattern with the highest probability in the likelihood calculator.

One exemplary data flows between a user with a cell phone or mobiledevice in an interactive conversation with third party devices ordoctors is discussed next. A patient is first registered with thesystem. After the user enrolls, the system starts communicating with thepatient by sending the patient one or more instructions and/orreminders. Using a computer such as a mobile device the usercommunicates with the physician communicator engine and receives inreturn a custom response. At the same time, and depending on selectedrules triggered by the patient response, the system sends notificationsto third-party devices such as devices owned by family members orcaregivers. The system can also send notifications to doctors, doctor'sstaff, or other authorized service providers who then send in responseresults that are automatically processed by the system to alter thebehavior of some rules.

FIG. 17A shows an exemplary process for automated interactivecommunication between clinicians and patients. The process includes codeto:

Set up rules for treatment modalities and assign zero or more rules toagent (3010)

Enroll patient and assign treatment modality to patient (3012)

During run time:

-   -   receiving communications from patients and selecting zero or        more agents to respond to the communication (3014)    -   receiving at zero or more event handlers messages from the zero        or more responsive agents and formats the messages for a target        device (3016)

FIG. 17B shows an exemplary process for applying the agents of FIG. 17Ato a weight loss treatment scenario. The general goals of weight lossand management are: (1) at a minimum, to prevent further weight gain;(2) to reduce body weight; and (3) to maintain a lower body weight overthe long term. The initial goal of weight loss therapy is to reduce bodyweight by approximately 10 percent from baseline. If this goal isachieved, further weight loss can be attempted, if indicated throughfurther evaluation. A reasonable time line for a 10 percent reduction inbody weight is 6 months of therapy. For overweight patients with BMIs inthe typical range of 27 to 35, a decrease of 300 to 500 kcal/day willresult in weight losses of about ½ to 1 lb/week and a 10 percent loss in6 months. For more severely obese patients with BMIs>35, deficits of upto 500 to 1,000 kcal/day will lead to weight losses of about 1 to 2lb/week and a 10 percent weight loss in 6 months. Weight loss at therate of 1 to 2 lb/week (calorie deficit of 500 to 1,000 kcal/day)commonly occurs for up to 6 months. After 6 months, the rate of weightloss usually declines and weight plateaus because of a lesser energyexpenditure at the lower weight.

After 6 months of weight loss treatment, efforts to maintain weight lossshould be put in place. If more weight loss is needed, another attemptat weight reduction can be made. This will require further adjustment ofthe diet and physical activity prescriptions.

Dietary Therapy: A diet that is individually planned and takes intoaccount the patient's overweight status in order to help create adeficit of 500 to 1,000 kcal/day should be an integral part of anyweight loss program. Depending on the patient's risk status, thelow-calorie diet (LCD) recommended should be consistent with the NCEP'sStep I or Step II Diet. Besides decreasing saturated fat, total fatsshould be 30 percent or less of total calories. Reducing the percentageof dietary fat alone will not produce weight loss unless total caloriesare also reduced. Isocaloric replacement of fat with carbohydrates willreduce the percentage of calories from fat but will not cause weightloss. Reducing dietary fat, along with reducing dietary carbohydrates,usually will be needed to produce the caloric deficit needed for anacceptable weight loss. When fat intake is reduced, priority should begiven to reducing saturated fat to enhance lowering of LDL-cholesterollevels. Frequent contacts with the practitioner during dietary therapyhelp to promote weight loss and weight maintenance at a lower weight.

An increase in physical activity is an important component of weightloss therapy, although it will not lead to substantially greater weightloss over 6 months. Most weight loss occurs because of decreased caloricintake. Sustained physical activity is most helpful in the prevention ofweight regain. In addition, it has a benefit in reducing cardiovascularand diabetes risks beyond that produced by weight reduction alone. Formost obese patients, exercise should be initiated slowly, and theintensity should be increased gradually. The exercise can be done all atone time or intermittently over the day. Initial activities may bewalking or swimming at a slow pace. The patient can start by walking 30minutes for 3 days a week and can build to 45 minutes of more intensewalking at least 5 days a week. With this regimen, an additionalexpenditure of 100 to 200 calories per day can be achieved. All adultsshould set a long-term goal to accumulate at least 30 minutes or more ofmoderate-intensity physical activity on most, and preferably all, daysof the week. This regimen can be adapted to other forms of physicalactivity, but walking is particularly attractive because of its safetyand accessibility. Patients should be encouraged to increase “every day”activities such as taking the stairs instead of the elevator. With time,depending on progress and functional capacity, the patient may engage inmore strenuous activities. Competitive sports, such as tennis andvolleyball, can provide an enjoyable form of exercise for many, but caremust be taken to avoid injury. Reducing sedentary time is anotherstrategy to increase activity by undertaking frequent, less strenuousactivities.

The communication system of FIG. 1A is used to provide Behavior Therapy.The system automatically sends messages using rule-based agents tocommunicate with patients. The agents can use learning principles suchas reinforcement provide tools for overcoming barriers to compliancewith dietary therapy and/or increased physical activity to help patientin achieving weight loss and weight maintenance. Specific communicationmessage include self-monitoring of both eating habits and physicalactivity, stress management, stimulus control, problem solving,contingency management, cognitive restructuring, and social supportthrough the social network system.

Pharmacotherapy can be used if behavior therapy does not work. Incarefully selected patients, appropriate drugs can augment LCDs,physical activity, and behavior therapy in weight loss. Drugs such assibutramine and orlistat can be used as long as potential side effectswith drugs are considered. With sibutramine, increases in blood pressureand heart rate may occur. Sibutramine should not be used in patientswith a history of hypertension, CHD, congestive heart failure,arrhythmias, or history of stroke. With orlistat, fat soluble vitaminsmay require replacement because of partial malabsorption. Weight losssurgery is one option for weight reduction in a limited number ofpatients with clinically severe obesity, i.e., BMIs>=40 or >=35 withcomorbid conditions. Weight loss surgery should be reserved for patientsin whom efforts at medical therapy have failed and who are sufferingfrom the complications of extreme obesity. Gastrointestinal surgery(gastric restriction [vertical gastric banding] or gastric bypass is anintervention weight loss option for motivated subjects with acceptableoperative risks. An integrated program must be in place to provideguidance on diet, physical activity, and behavioral and social supportboth prior to and after the surgery.

The agents are adaptive to the patient and allow for programmodifications based on patient responses and preferences. For example,the agent can be modified for weight reduction after age 65 to addressrisks associated with obesity treatment that are unique to older adultsor those who smoke.

FIG. 17C shows in more details the event handler of FIG. 17A. Theprocess includes code to:

Receive message from patient or doctor (3020)

Determine user treatment modality (3022)

For each modality

-   -   Determine relevant rules (3026)    -   For each rule        -   Determine responsive agent(s) (3030)        -   For each agent            -   Execute agent program (3034)            -   Get input from service provider if needed (3036)            -   Format & send the message for the patient's mobile                device (3038)

The system processes a communication from a patient according to one ormore treatment scenarios. Each treatment scenario is composed of one ormore rules to be processed in a sequence that can be altered wheninvoking certain agents.

The if-then rules can be described to the system using a graphical userinterface that runs on a web site, a computer, or a mobile device, andthe resulting rules are then processed by a rules engine. In oneembodiment, the if-then rules are entered as a series of dropdownselectors whose possible values are automatically determined andpopulated for user selection to assist user in accurately specifying therules.

In one embodiment, the rules engine is Jess, which is a rule engine andscripting environment written entirely in Sun's Java language by ErnestFriedman-Hill at Sandia National Laboratories in Livermore, Calif. anddownloadable at http://www.jessrules.com/jess/index.shtml. With Jess,the system can “reason” using knowledge supplied in the form ofdeclarative rules. Jess is small, light, and one of the fastest ruleengines available. Jess uses an enhanced version of the Rete algorithmto process rules. Rete is a very efficient mechanism for solving thedifficult many-to-many matching problem (see for example “Rete: A FastAlgorithm for the Many Pattern/Many Object Pattern Match Problem”,Charles L. Forgy, Artificial Intelligence 19 (1982), 17-37.) Jess hasmany unique features including backwards chaining and working memoryqueries, and of course Jess can directly manipulate and reason aboutJava objects. Jess is also a powerful Java scripting environment, fromwhich Java objects, call Java methods, can implement Java interfaceswithout compiling any Java code.

The user can dynamically create an if/then/else statement. A dropdownselector can be used to select a column, then a dropdown to select theconditional operator (=, >, <, !=, among others) and then a text box inwhich to enter a column, text or number value. The system can addmultiple conditions. The rules can be saved as serialized object in adatabase. After entering parameter values, a new set of rules can begenerated and inserted within the current active scenario. Thecorresponding rules can then be modified directly by accessing theindividual agents within the rules.

In one embodiment, the agent can be self-modifying. The agent receivesparameters from its callers. The agent in turn executes one or morefunctions. It can include an adaptive self-modifying function, and thethird-party extension interfaces. The adaptive self-modifying functionis capable of modifying the agent parameters and/or the agent functionat run time, thereby changing the behavior of the agent.

FIG. 17D shows an exemplary modality of the rules engine to serve obesepatients that the doctor can review and approve. In this scenario, theengine executes 3 master agents:

Run blood pressure master agent (3050)

Run diabetic master agent (3052)

Run weight loss agent (3054).

The blood pressure master agent in turn invokes the following agents:

If blood pressure is between 130-139/85-89 mm Hg then run agenthigh_blood_pressure

If blood pressure is between 140-159/90-99 mm Hg then run agentstage1_blood_pressure

If blood pressure is above 159/99 mm Hg then run agentdrug_treatment_for_blood_pressure

For the above example, high normal blood pressure of between130-139/85-89 mm Hg is included in the risk stratification. In patientswith high normal blood pressure with no or only one concurrent riskfactor that does not include diabetes, target organ, or clinical cardiacdisease, the agent high_blood_pressure suggests to the patient to uselifestyle modification to lower blood pressure. Lifestyle modificationincludes changes to the patient's dieting and exercising habits. With arisk factor of target organ or clinical cardiac disease, diabetes and/orother risk factors, the agent can recommend drug therapy, no matter whatthe patient's blood pressure is. The agent for patients with stage 1blood pressures of between 140-159/90-99 mm Hg who have no other riskfactors will suggest the patient try lifestyle modifications for a yearbefore drug therapy is used. But if these patients have one risk factorother than diabetes, target organ, or clinical cardiac disease, theirlifestyle modification should be tried for only 6 months beforeinitiation therapy. For patients with blood pressure above 150/100 mmHg, the agent reminds the patient to have drug therapy in addition tolifestyle modifications.

The diabetic master agent in turn invokes the following agents:

-   -   Monitoring agent: Make sure doctor orders the key tests at the        right times.    -   Dieting planning agent: Work with a dietitian to develop a great        eating plan.    -   Glucose Testing Agent: Check blood glucose at correct intervals.    -   Exercise agent: Monitor exercise to help heart.    -   Medication compliance agent: check that insulin is taken at        correct time.    -   Foot care agent: Check your feet with your eyes daily.    -   Eye care agent: remind patient to get periodic eye exam.

The weight loss agent considers the patient's BMI, waist circumference,and overall risk status including the patient's motivation to loseweight. The weight loss agent in turn calls the following agents:

Body Mass Index agent: The BMI, which describes relative weight forheight, is significantly correlated with total body fat content. The BMIshould be used to assess overweight and obesity and to monitor changesin body weight. In addition, measurements of body weight alone can beused to determine efficacy of weight loss therapy. BMI is calculated asweight (kg)/height squared (m2). To estimate BMI using pounds andinches, use: [weight (pounds)/height (inches)2]×703. Weightclassifications by BMI, selected for use in this report, are shownbelow:

CLASSIFICATION OF OVERWEIGHT AND OBESITY BY BMI Obesity Class BMI(kg/m²) UnderWeight <18.5 Normal 18.5-24.9 Overweight 25.0-29.9 ObesityI 30.0-34.9 II 35.0-39.9 Extremely Obesity III 240A conversion table of heights and weights resulting in selected BMIunits is

SELECTED BMI UNITS CATEGORIZED BY INCHES (CM) AND POUNDS (KG) BMI 25kg/m² BMI 25 kg/m² BMI 25 kg/m² Height in inches (cm) Body Weight inpounds (kg) 58 (147.32) 119 (53.98) 129 (58.81) 143 (64.86) 59 (149.86)124 (56.25) 133 (60.33) 148 (67.13) 60 (152.40) 128 (58.06) 138 (62.60)153 (69.40) 61 (154.94) 132 (59.87) 143 (64.86) 158 (71.67) 62 (157.48)136 (61.69) 147 (66.68) 164 (74.39) 63 (160.02) 141 (63.96) 152 (68.95)169 (76.66) 64 (162.56) 145 (68.77) 157 (71.22) 174 (78.93) 65 (165.10)150 (68.04) 162 (73.48) 180 (81.65) 66 (167.54) 155 (70.31) 167 (75.75)186 (84.37) 67 (170.18) 159 (72.12) 172 (78.02) 191 (86.64) 68 (172.72)164 (74.39) 177 (80.29) 197 (89.36) 69 (175.26) 169 (76.66) 182 (82.56)203 (92.08) 70 (177.80) 174 (78.93) 188 (85.28) 207 (93.90) 71 (180.34)179 (81.19) 193 (87.54) 215 (97.52) 72 (182.88) 184 (83.46) 199 (90.27) 221 (100.25) 73 (185.42) 189 (88.73) 254 (92.53)  227 (102.97) 74(187.96) 194 (88.00) 210 (95.26)  233 (105.69) 75 (190.50) 200 (90.72)216 (97.98)  240 (108.86) 76 (193.04) 205 (92.99)  221 (100.25)  246(111.59) Metrix conversion formula = Non-metric conversion formula =Weight (kg)/height (m)² [weight (pounds)/height (inches)²] × 704.5Example of BMI calculation: Example of BMI calculation: A person whoweighs 78.93 A person who weighs 164 ponds and is kilograms and is 17768 inches (or 5′8′) tall has a BMI of 25. centimeter tall has BMI[weight (164 ponds)/height (68 inches)²] × of 25. weight (78.93 kg)/704.5 = 25 height (1.77 cm)² = 20.

Waist Circumference agent: The presence of excess fat in the abdomen outof proportion to total body fat is an independent predictor of riskfactors and morbidity. Waist circumference is positively correlated withabdominal fat content. It provides a clinically acceptable measurementfor assessing a patient's abdominal fat content before and during weightloss treatment. The sex-specific cutoffs noted on the next page can beused to identify increased relative risk for the development ofobesity-associated risk factors in most adults with a BMI of 25 to 34.9kg/m2: These waist circumference cutpoints lose their incrementalpredictive power in patients with a BMI>=35 kg/m2 because these patientswill exceed the cutpoints noted above. The disease risk of increasedabdominal fat to the disease risk of BMI is as follows:

CLASSIFICATION OF OVERWEIGHT AND OBESITY BY BMI, WAIST CIRCUMFERENCE ANDASSOCIATED DISEASE RISKS Disease Risk* Relative to Normal Weight andWaist Circumference Obesity Men ≦ 102 cm (≦86 in) >102 cm (>86 in) BMI(kg/m²) Class Women ≦ 88 cm (≦35 in ) >88 cm (>35 in) Under Weight <18.5— — Normal 18.5-24.9 — — Overweight 25.0-29.9 Increased High Obesity30.0-34.9 I High Very High 35.0-38.9 II Very High Very High ExtremelyObesity ≧40 III Extermely High Extermely High

These categories denote relative risk, not absolute risk; that is,relative to risk at normal weight. They should not be equated withabsolute risk, which is determined by a summation of risk factors. Theyrelate to the need to institute weight loss therapy and do not directlydefine the required intensity of modification of risk factors associatedwith obesity.

Risk Status agent is used for assessment of a patient's absolute riskstatus and in turn uses the following agents:

-   -   1) Disease condition agent: determine existence of coronary        heart disease (CHD), other atherosclerotic diseases, type 2        diabetes, and sleep apnea.    -   2) Obesity-associated disease agent: determines gynecological        abnormalities, osteoarthritis, gallstones and their        complications, and stress incontinence.    -   3) Cardiovascular risk factors agent: cigarette smoking,        hypertension (systolic blood pressure>=140 mm Hg or diastolic        blood pressure>=90 mm Hg, or the patient is taking        antihypertensive agents), high-risk LDL-cholesterol (>=160        mg/dL), low HDL-cholesterol (<35 mg/dL), impaired fasting        glucose (fasting plasma glucose of 110 to 125 mg/dL), family        history of premature CHD (definite myocardial infarction or        sudden death at or before 55 years of age in father or other        male first-degree relative, or at or before 65 years of age in        mother or other female first-degree relative), and age (men>=45        years and women>=55 years or postmenopausal). Patients can be        classified as being at high absolute risk if they have three of        the aforementioned risk factors. Patients at high absolute risk        usually require clinical management of risk factors to reduce        risk. Patients who are overweight or obese often have other        cardiovascular risk factors. Methods for estimating absolute        risk status for developing cardiovascular disease based on these        risk factors are described in detail in the National Cholesterol        Education Program's Second Report of the Expert Panel on the        Detection, Evaluation, and Treatment of High Blood Cholesterol        in Adults (NCEP's ATP II) and the Sixth Report of the Joint        National Committee on Prevention, Detection, Evaluation, and        Treatment of High Blood Pressure (JNC VI). The intensity of        intervention for cholesterol disorders or hypertension is        adjusted according to the absolute risk status estimated from        multiple risk correlates. These include both the risk factors        listed above and evidence of end-organ damage present in        hypertensive patients. Approaches to therapy for cholesterol        disorders and hypertension are described in ATP II and JNC VI,        respectively. In overweight patients, control of cardiovascular        risk factors deserves equal emphasis as weight reduction        therapy. Reduction of risk factors will reduce the risk for        cardiovascular disease whether or not efforts at weight loss are        successful.

Other risk factors can be considered as rules by the agent, includingphysical inactivity and high serum triglycerides (>200 mg/dL). Whenthese factors are present, patients can be considered to haveincremental absolute risk above that estimated from the preceding riskfactors. Quantitative risk contribution is not available for these riskfactors, but their presence heightens the need for weight reduction inobese persons.

A patient motivation agent evaluates the following factors: reasons andmotivation for weight reduction; previous history of successful andunsuccessful weight loss attempts; family, friends, and work-sitesupport; the patient's understanding of the causes of obesity and howobesity contributes to several diseases; attitude toward physicalactivity; capacity to engage in physical activity; time availability forweight loss intervention; and financial considerations. In addition toconsidering these issues, the system can heighten a patient's motivationfor weight loss and prepare the patient for treatment through normativemessaging and warnings. This can be done by enumerating the dangersaccompanying persistent obesity and by describing the strategy forclinically assisted weight reduction. Reviewing the patients' pastattempts at weight loss and explaining how the new treatment plan willbe different can encourage patients and provide hope for successfulweight loss.

The agents facilitate the interaction between doctors and patients usingdevices such as smartphones, tablets, and emerging wearable devices, forthe creation of new, powerful, easy-to-use solutions in healthcare. Itis particularly well suited for applications using popular and emerginguser and machine-to-machine interaction technologies such as instantmessaging, emails, SMS, near field communications (NFC), and M2M.

FIG. 18A shows another exemplary process for monitoring a patient withthe agents. The process starts with patient registration (3100) andcollection of information on patient (3102). Next, the process selects atreatment template based on treatment plan for similar patients (3104).The process generates a treatment plan from the template and customizesthe treatment plan (3006). The system considers the following factors:medical condition, amount of weight to lose, physician observationsregarding mental state of the patient.

In the event the patient has extensive or contraindicating medicalhistory or information, the system alerts the doctor to manually reviewthe patient file and only generate recommendations with authorizationfrom a doctor.

The doctor subsequently reviews and discusses the customized plan withthe patient. In one embodiment, during the discussion, the doctor offersthe patient the opportunity to enroll in the automated monitoringprogram. For a monthly or yearly fee, the system would provide thepatient with periodic encouragements or comments from the system or thephysician. In one embodiment, the doctor can provide the patient with anoptional monitoring hardware that measures patient activity (such asaccelerometers) and/or vital signs (such as EKG amplifiers).

Upon user enrollment, the system's workflow helps the doctor withsetting goals with the patient, establishing a bond of trust andloyalty, and providing positive feedback for improving compliance.Loyalty to the practitioner initially produces higher compliance,emphasizing that establishing a close relationship helps. By providingrapid feedback through instant messaging or emails, the system helpsdoctors earn the patient's respect and trust, set goals together withthe patient, and praise progress when it occurs.

Once enrolled, the system collects data on patient compliance with atreatment plan (3008). This can be done using mobile devices withsensors such as MEMS devices including accelerometer and others asdescribed more fully below. Alternatively, the system periodicallyrequests patient data will be weighed, measured, body fat calculated,blood pressure, resting heart rate and overall well-being. In oneembodiment, the system provides a daily (7 days a week) counselingprocess using texting, email or social network communications.

The process also accumulates reward points for patient to encouragehealthy activities, such as jogging, walking, or gardening (3110). Theprocess also compares patient progress with other patients (3112) andsends automatic encouraging messages to patients (3114). Upon patientauthorization, the system announces the patient's goals and progress toa social network such as Facebook. The social network strengthens thepatient's will for dieting and exercise by the “extent to whichindividuals perceive that significant others encourage choice andparticipation in decision-making, provide a meaningful rationale,minimize pressure, and acknowledge the individual's feelings andperspectives.” The system supplements the treatment through socialsupports at home and encourages the patient to make their family andclose friends aware of their condition and the expectations of diet andexercise. This will provide the patient with encouragement andaccountability.

Periodically, the system shows patient status to doctor (3016) andpresents recommendations to doctor on preventive steps, such ascheck-ups and basic blood tests (3018). Automatically, the systemschedules in person consultation for patient and doctor (3020). Capturedprogress data can be viewed by the physicians and patients using a webbased system. The physician can review all interactions between thesystem and the patient. The physician is able to see their progressreports, interactive e mail which includes daily menus and notes betweenthe service and the patient. The physician will be able to check on thepatient's progress at any time of day or night. The system improves theDoctor-Patient relationship and influences compliance.

The system's interactive behavior combines four key elements:just-in-time information, automation in checking with patients,persuasive techniques or messaging, and user control elements. In oneembodiment, reports about the user's calorie consumption and exerciseactivity over time, and in comparison to similarly situated people, aregenerated.

The system provides meaningful feedback, allowing customers to “see”their food consumption, exercise and the impact of changes. Whencalories from eating go up between months, a graph depicts so and by howmuch. Without the system's report to conveniently compare foodconsumption and exercise from one week to the next, it would be muchharder to track those changes. Feedback provides the information crucialto bring about self-awareness of one's actions.

Additionally, the greater value of the system is that it provides usefulinformation about what other similar users' actions and impacts arelike. The report shows where the patient's energy intake and outtake arein comparison to the healthiest and the average person. This informationserves as a descriptive norm, letting customers know where they are inthe spectrum of average and healthy people. When customers see that theyare below or even just above average, they want to move “up” on theexercise but reduce their calorie intake. As humans, users areprogrammed to want to be unique . . . but not too unique—they want tohave “normal” food consumption and normal health.

With regard to the message persuasiveness, content is positive andtargeted to the user's specific situation. The system provides actionopportunities with its reports. If the user is mildly overweight, itmight offer a suggestion of having salad with a low calorie dressing fordinner. One embodiment provides a “marketplace” concept, which meansthat the suggestion would be accompanied by, say, a coupon for salad ata local restaurant. In one embodiment, the system has priorrelationships with partners such as restaurants that would offer mealswith preset calorie and can send the user coupons to different partnerson different days, thus providing users with a wide range of healthyfood selections. The system's power lies in its ability tosimultaneously prep individuals for action and give them an easyopportunity to do so.

In sum, the system's feedback is effective because:

-   -   It is provided frequently, as soon after the consumption        behavior as possible.    -   It is clearly and simply presented.    -   It is customized to the patient's specific medical condition.    -   It is provided relative to a meaningful standard of comparison.    -   It is provided over an extended period of time.    -   It includes specific food consumption and calorie breakdown.    -   It is interactive through instant messaging, email, or social        networks.

In one embodiment, body analysis data is determined from enrollmentdata, and include: body mass ratio, pounds of lean muscle mass,percentage of body fat and an optimal range for the specific individualof that percentage, pounds of body fat and an optimal range of body fatfor that specific individual, and suggested pounds of body fat to lose.The body analysis includes the following: Basal Metabolic Rate (BMR) isthe number of calories burned by the patient's lean body mass in a 24hour period at complete rest using formulas such as the Harris-Benedictformula or other suitable formulas. Specific Dynamic Action of Foods(SDA) is the numbers of calories required to process and utilizeconsumed foods (in one case estimated at 5-15% of BMR, depending onpersonalization). Resting Energy Expenditure (REE) is the sum of BMR andSDA and represents the number of calories that the patient's bodyrequires in a 24 hour period at complete rest. The system determines aProgram Recommendation Total Caloric Intake as the caloric supplementrequired to achieve weight loss of approximately 2 pounds per week.Medications or stimulating substances (such as caffeine, gingsen, ordiethylpropion) to assist in weight loss may be recommended and if sothe program increases calorie consumption based on a model of thepatient's response to such substances.

In one embodiment using the optional mobile monitoring hardware, thesystem determines Activities of Daily Living (ADL) as the number ofcalories burned by the patient's body during normal daily activitiesusing accelerometers. The accelerometers can also determine the CaloriesBurned by Exercise as the number of calories burned by the exercisesselected by the patient. Also included, is the level and intensity ofthe patient's activities. In one embodiment without the optional mobilemonitoring hardware, the system approximates the Activities of DailyLiving (ADL) as an average of calories expected to be burned by thepatient's body during normal daily activities, and in one case isestimated at 20% or REE. The system can also receive averagedapproximations of Calories Burned by Exercise is the number of caloriesburned by the exercises selected by the patient. Also included, is thelevel and intensity of the patient's activities.

FIG. 18B shows an exemplary process for monitoring patient food intake.The process first determines and recommends optimal diet based onpatient parameters (3130). To monitor progress, the process takes userentered calorie data and optionally captures images of meals using amobile device such as a mobile camera (3132). The process thentranslates images of the meals into calories (3134). The patient'sactual diet is then compared to with the recommended diet (3136).

In one embodiment, the camera captures images of the food being servedto the patient. The image is provided to an image search system such asthe Google image search engine, among others. The search returns thelikely type of food in the dish, and an estimation of the containervolume is done. In one embodiment, the volume can be done using a 3Dreconstruction using two or more images of the food found as theintersection of the two projection rays (triangulation). The two imagesfrom the 2D images are selected to form a stereo pair and from densesets of points, correspondences between the two views of a scene of thetwo images are found to generate a 3D reconstruction is done to estimatethe 3D volume of each food item.

The system determines and looks up a database that contains calorie perunit volume for the dish being served, and multiplies the food volumeestimate with the calorie per unit volume for the type of food to arriveat the estimated total calorie for the dish. The user is presented withthe estimate and the details of how the estimation was arrived at areshown so the user can correct the calorie estimation if needed.

FIG. 18C shows an exemplary exercise recommendation and monitoringprocess. First, the process determines and recommends an exerciseroutine that is customized to the patient's medical condition (3140).The process then captures patient exercise activity usingmicro-electromechanical systems (MEMS) sensors (3142). The MEMS sensorscan include Accelerometer, Gyroscope, Magnetometer, Pressure sensor,Temperature, and Humidity sensor, among others. The process thencorrelates actual patient activity with the recommended exercises(3144).

In one embodiment, the above process collects data from and thencompares the performance of the patient with similar patients. Theprocess engages and motivates through Social Network Encouragement.

In another embodiment, initial data captured from the patient includesblood pressure, chronic diseases, present weight, height, presentphysical health, whether the patient is diabetic, among others. Othermeasurements include height, weight, blood pressure, as well asmeasurements of the wrist, upper arms, chest, waist, hips, thighs andfinger. In preparing an individual weight loss program it is essentialto consider the present physical activity level and diet of the patient.

In yet another embodiment, the system can automatically generate graphsfor review by the patient and the dietitian to aid in preparing a mealplan and for the benefit of the patients to aid in setting goals inaltering the eating habits. One chart can be a nutrient chart showingrecommended daily requirements with respect to the amount of vitaminsand nutrients consumed during an average day in comparison with therecommended allowance. The graph thus gives a clear indication ofdeficiencies and excessive amounts of vitamins and nutrients consumed inthe present diet. Another chart shows calories from fats, protein,carbohydrates and alcohol in comparison with a similar chart ofrecommended percentages of calorie intake to show deficiencies in thesource of calories and the need to reduce calorie intake from othersources.

Another chart can show consistency of the patient and potentialcorrective actions to existing eating habits. An additional column liststhe Recommended Daily Allowance as recommended by the U.S. SenateCommittee. From this data the percentage of nutrient intake compared tothe Recommended Daily Allowance is calculated and printed on the report.Finally a column reporting the deficiency or excess of nutrientsconsumed appears. From this column the dietitian can readily identifythose nutrients of which the patient's diet suffers a deficiency.

In determining the percentages of nutrients compared to the recommendeddaily allowance the computer goes through a series of determinationsusing fiber intake as an example. From the actual diet the computer isable to determine the amount of fiber in each food item. The generallyaccepted range of fiber intake is from 10-40 grams per day. The computerinitially determines if the daily fiber intake is in excess of themaximum 40 grams/day allowance. If the fiber intake does not exceed 40grams the system determines the percentage fiber over the maximumallowed amount. If the fiber intake does not exceed 40 grams/day thecomputer then determines if the fiber intake is greater than 10grams/day. If the fiber intake is greater than 10 grams/day and lessthan 40 grams/day the computer will indicate that the fiber intake iswithin the acceptable level. Should the fiber intake be less than 10grams/day the total fiber intake is divided by 10 to calculate thepercentage of fiber in the daily diet. A similar analysis is performedby the computer for each of the nutrients.

One embodiment provides a meal plan based on the American DiabeticsAssociation recommendation which breaks food down into 8 categories.These categories include a fruit exchange, vegetable exchange, milkexchange, grain and cereal exchange, fat exchange, lean meat exchange,medium fat meat exchange, high fat meat exchange. Under each of thesecategories a wide variety of food items is listed which offers thepatient a selection of food items according to their taste preferences.While the total calorie consumption per day is of major importance, itis essential that these calories be provided from the sources specifiedin the meal plan. Once the patient has designated a meal plan thepatient can select a food item from each of the specified foodexchanges. This allows variation and compensation for the patient's foodpreferences.

The exercise chart provided educates the patient as to what activitiesshould be considered, proper methods of selecting an exercise programand proper procedures of exercising. Additionally, the exercise chartprovides training guidelines for increasing one's exercise through a 12week period. For each week of the training period the exercise chartprovides guidelines as to duration and level of physical activity. Anaccelerometer is used to monitor the duration and level of activity.

Additional measurements 6 taken at this time are of fundamentalimportance in having a significant psychological impact on the patient.From these measurements the patient is able to readily recognize theneed for a change in his eating habits and lifestyle. Throughout theweight loss program these measurements provide evidence of a reductionin size of various parts of the body and in addition to a reduction inweight gives the patient encouragement and confidence that the programis effective. The measurements include the circumference of the arms,waist, hips, chest and fingers.

The next stage in the weight management system involves extensiveconsultation between the patient, the system's communication agents, andthe dietitian using the communication facilities of the system. Thesystem, with the dietician, automatically outlines the deficiencies ofthe patient's existing eating habits and diet compared to a breakdown ofcalorie intake as recommended by the United States Senate Committee. Thesystem shows a comparison chart with expert recommendations and thepatient's own diet results in a higher interest which in turn makeslearning much easier.

Initially the system helps the patient to correct one dietary correctionas the goal for that week. At the end of the week the rule-based agentreviews the progress and if that one weakness is showing satisfactoryimprovement, additional goals are set to correct the remainingdeficiencies. Normally the deficiencies are corrected one at a time. Inorder for the dietitian to aid the patient in setting goals the computeranalysis calculates at which nutrients a deficiency exists. In oneembodiment, the rule based agent classifies the nutrient into twocategories identified as Priority I and Priority II nutrients. ThePriority I nutrients are the nutrients where the amount taken in must bewithin a specified range. The human body requires a specified amount ofthese nutrients to function properly on a daily basis, however, anexcessive amount has been shown to have undesirable consequences. ThePriority I nutrients for the purposes of this example include,cholesterol, fiber, sodium fat and calcium. The Priority II nutrientsare those which, while essential to good health, are not of majorconcern if the diet periodically does not provide the recommended dailyallowance. For the purposes of example only these Priority II nutrientsmay include vitamin A, riboflavin, niacin and vitamin C. The computeranalysis determines whether there is a deviation of any of thesePriority I nutrients from the recommended daily allowance. If there is adeviation the computer determines the number of nutrients of which thereis a deviation and if there are two or more the computer selects the twonutrients with the highest deviation. From this identification thedietition is able to recommend particular foods to correct the nutrientexcess or deficiency. For example, if the analysis indicated the sodiumand fat level as being abnormally high the dietition might recommend areduction in specific types of meat and reducing the intake of saltedfoods. The two highest deviations would normally be used as the firstgoals.

If there is only one Priority I nutrient deviation it has beenconsidered not to be of major immediate concern and will normally becorrected by adjusting the overall diet. The computer detecting only onedeviation of Priority I nutrients then determines the number ofdeviation from the recommended daily allowance of the Priority IInutrients. As with the Priority I nutrients the computer identifies thetwo nutrients having the greatest deviation. From this identificationthe system is able to set goals with the patient to correct thedeviation. For example, if the computer determines this is a majordeficiency of vitamin C and vitamin A the rule based agent mightrecommend a diet concentrating more on green vegetables and citrusfruits.

In selecting a meal plan it is necessary to determine the number ofcalories recommended per day in order to lose and maintain a desiredweight. The first step is to find the patient's reasonable or desiredweight for their height and build. From the height and build the weightis determined based on data prepared by the Metropolitan InsuranceCompany in 1983. The Handbook of Clinical Dietetics, Yale UniversityPress, provides a formula for determining the maximum calories to beconsumed per day necessary to maintain one's weight.

For those extremely overweight a drastic reduction in calorie intake isliable to have serious and psychological and emotional consequenceswhich may rapidly discourage the patient and leave the program. For thistype of situation an intermediate menu plan is prepared by computeranalysis to offer a compromise between the ideal calorie intake level toachieve the desired weight and the calorie intake of the old eatinghabits. In determining whether an intermediate menu plan is appropriatethe computer receives an input of the present weight A, ideal weight Band the activity level C. The activity level is the number of caloriesneeded per pound per day. From this data the computer determines theexcess in calories consumed per day D according to the formula:(A×C)−(B×C)=D If the excess calories D is greater than 1000 calories perday an intermediate menu plan is recommended.

The system also encourages social network based behavior modificationwith friends and family and normative message that compare the patient'sprogress with similar patients. Acceptance of the meals by the familymembers offers positive reinforcement to the patients which have provento be a powerful tool in changing one's lifestyle. For most people theweight management system is a tremendous undertaking which requiressupport from the patient's family. It has been found that familycounseling can be a major factor in the success of the program.Effective weight loss and change in lifestyle affects not only anindividual but everyone in the patient's environment. Support isnecessary from the family members in order to maximize the chances ofsuccess.

Using the social network messaging, positive reinforcement is furtherprovided by group sessions between the different patients. Each patientasks questions, exchanges experiences and offers support andencouragement to each other. Discussions about the reactions of familymembers to the meals prepared offers encouragement to others to trydifferent recipes. There are two types of social norms that exertinfluence. While the result, conformity, is the same for both types ofinfluence, the motivation behind conformity is different in each case.Injunctive norms encourage conformity by implying that a certainattitude or behavior is approved of or disapproved of by a social group.Descriptive norms imply that an attitude or behavior is common amongmembers of a group, regardless of approval.

Exercise is an essential part of any effective weight management system.A group sessions by a fitness trainer or physiologist on the benefitsand attributes of proper exercise can be scheduled. Techniques forproper ways to exercise and the proper amounts of exercise are discussedin detail. Various activities by the individual patients are shared withthe others thereby offering encouragement and reinforcement for theirgoals of managing their weight effectively in a safe and healthy manner.Different exercise programs and activities are discussed and dates areset for these activities. It has been found that for the beginner a firmcommitment by the patient is an essential aspect of developing a regularexercise program.

The normative messaging relies on normative social influence which is“the influence of other people that leads us to conform in order to beliked and accepted by them.” This often leads to public compliance—butnot necessarily private acceptance—of the group's social norms. Whenpeople tend to conform to normative social influence is explained by thesocial impact theory. Social impact theory states that the moreimportant the group is, the closer the physical distance is between thegroup and oneself, and the number of people in the group all affect thelikelihood that one will conform to the group's social norms. This issimilar to one environment where households that received more normativemessages in which described the frequency and amount of weeklyrecycling, began to have a direct impact on both the householdsfrequency and amount of curbside recycling. The sudden change was due tothe fact that “the other neighbors” recycling habits had a directnormative effect on the household to change theirs. Similar results wereapparent in hotels where towel usage increased by 28% through normativemessages. Direct personal experience is not a necessity for normativetendency. Written communication instructing how people should behave ordescribing how most people act in a given situation can generate thesame normative behavior in people.

In many cases, normative social influence serves to promote socialcohesion. When a majority of group members conform to social norms, thegroup generally becomes more stable. This stability translates intosocial cohesion, which allows group members to work together toward acommon understanding, or “good”.

As part of the exercise session a specific activity is scheduled andundertaken by the patients which may for example be a walk or muscleexercises. FIG. 19 shows an exemplary kinematic exercise monitoringsystem. In one implementation, an HMM is used to track patient movementpatterns. Human movement involves a periodic motion of the legs. Regularwalking involves the coordination of motion at the hip, knee and ankle,which consist of complex joints. The muscular groups attached at variouslocations along the skeletal structure often have multiple functions.The majority of energy expended during walking is for vertical motion ofthe body. When a body is in contact with the ground, the downward forcedue to gravity is reflected back to the body as a reaction to the force.When a person stands still, this ground reaction force is equal to theperson's weight multiplied by gravitational acceleration. Forces can actin other directions. For example, when we walk, we also produce frictionforces on the ground. When the foot hits the ground at a heel strike,the friction between the heel and the ground causes a friction force inthe horizontal plane to act backwards against the foot. This forcetherefore causes a breaking action on the body and slows it down. Notonly do people accelerate and brake while walking, they also climb anddive. Since reaction force is mass times acceleration, any suchacceleration of the body will be reflected in a reaction when at leastone foot is on the ground. An upwards acceleration will be reflected inan increase in the vertical load recorded, while a downwardsacceleration will be reduce the effective body weight. Wireless sensorswith tri-axial accelerometers are mounted to the patient on differentbody locations for recording. Sensors can be placed on (or cameras candetect) the four branches of the links connect to the root node (torso)with the connected joint, left shoulder (LS), right shoulder (RS), lefthip (LH), and right hip (RH). Furthermore, the left elbow (LE), rightelbow (RE), left knee (LK), and right knee (RK) connect the upper andthe lower extremities. The wireless monitoring devices can also beplaced on upper back body near the neck, mid back near the waist, and atthe front of the right leg near the ankle, among others.

The sequence of human motions can be classified into several groups ofsimilar postures and represented by mathematical models calledmodel-states. A model-state contains the extracted features of bodysignatures and other associated characteristics of body signatures.Moreover, a posture graph is used to depict the inter-relationshipsamong all the model-states, defined as PG(ND,LK), where ND is a finiteset of nodes and LK is a set of directional connections between everytwo nodes. The directional connection links are called posture linksEach node represents one model-state, and each link indicates atransition between two model-states. In the posture graph, each node mayhave posture links pointing to itself or the other nodes.

In the pre-processing phase, the system obtains the human body profileand the body signatures to produce feature vectors. In the modelconstruction phase, the system generates a posture graph, examinefeatures from body signatures to construct the model parameters of HMM,and analyze human body contours to generate the model parameters ofASMs. In the motion analysis phase, the system uses features extractedfrom the body signature sequence and then applies the pre-trained HMM tofind the posture transition path, which can be used to recognize themotion type. Then, a motion characteristic curve generation procedurecomputes the motion parameters and produces the motion characteristiccurves. These motion parameters and curves are stored over time, andcalorie consumption is assigned to each muscle movement and can bevaried according to the frequency and the range of the movement toarrive at an accurate measurement of energy expenditure when the patientwalks or exercises.

For most healthy adults, the Department of Health and Human Servicesrecommends these exercise guidelines:

Aerobic activity. Get at least 150 minutes a week of moderate aerobicactivity or 75 minutes a week of vigorous aerobic activity. However, toeffectively lose or maintain weight, some people may need up to 300minutes a week of moderate physical activity. You also can do acombination of moderate and vigorous activity. The guidelines suggestthat you spread out this exercise during the course of a week, andsessions of activity should be at least 10 minutes in duration.

Strength training Do strength training exercises at least twice a week.No specific amount of time for each strength training session isincluded in the guidelines.

Moderate aerobic exercise includes such activities as brisk walking,swimming and mowing the lawn. Vigorous aerobic exercise includes suchactivities as running and aerobic dancing. Strength training can includeuse of weight machines, or activities such as rock climbing or heavygardening.

As a general goal, the system encourages the user to perform at least 30minutes of physical activity every day. Specific calorie expendituresvary widely depending on the exercise, intensity level and the user'sindividual situation. The system receives communications from the useron the exercise, and uses rules from the chart below on the estimatednumber of calories burned while doing various exercises for one hour, asmodified using the MEMS activity sensor and the kinematics model of FIG.19.

Weight of person and calories burned 200 pounds Activity 160 pounds (91240 pounds (1-hour duration) (73 kilograms) kilograms) (109 kilograms)Aerobics, high impact 533 664 796 Aerobics, low impact 365 455 545Aerobics, water 402 501 600 Backpacking 511 637 736 Basketball game 584728 872 Bicycling, <10 mph, 292 364 436 leisure Bowling 219 273 327Canoeing 256 319 382 Dancing, ballroom 219 273 327 Football, touch orflag 584 728 872 Golfing, carrying clubs 314 391 469 Hiking 438 546 654Ice skating 511 637 763 Racquetball 511 637 763 Resistance (weight) 365455 545 training Rollerblading 548 683 818 Rope jumping 861 1,074 1,286Rowing, stationary 438 546 654 Running, 5 mph 606 755 905 Running, 8 mph861 1,074 1,286 Skiing, cross-country 496 619 741 Skiing, downhill 314391 469 Skiing, water 438 546 654 Softball or baseball 365 455 545 Stairtreadmill 657 819 981 Swimming, laps 423 528 632 Tae kwon do 752 9371,123 Tai chi 219 273 327 Tennis, singles 584 728 872 Volleyball 292 364436 Walking, 2 mph 204 255 305 Walking, 3.5 mph 314 391 469

The system can automatically schedule appoints for doctors, dieticians,and fitness consultants. One exemplary platform supports multi-vendorscheduling with the rule based communication system discussed above. Theplatform provides a one-stop scheduling system for end-users to makeappointments and connect with medical service providers who sign up withthe system. For example, the scheduling platform serves a variety ofbusiness verticals such as doctor offices, specialist offices 4104,hospitals, dieticians, and exercise specialists, among others. Thesystem provides a web-based and mobile scheduling software forconnecting multiple industries' scheduling onto one platform.

The system minimizes the hassle of booking appointments through an arrayof channels with no consistencies or simplicity: phone, online, bookingsites, hospital or medical office websites. The system reduces errorarising when the user forgets to put the scheduled appointment ontocalendar (iPhone, Outlook, Google). The system reduces the time andeffort required to find a service provider with walk-in availabilitygiven an impromptu desire. Additionally, the inefficiency of manualscheduling of appointments and staff availability is avoided.

A user can sign-up with the system at a web site or download through anapp directly on iPhone or Android-operated phones. For convenience, thesystem allows the use of Facebook sign-in information. Once the user issigned-in, the user can search for a specific company or by certaincriteria (type of service, closest date of availability, etc), andschedule the appointment. The appointment will also integrate with theuser's choice of major calendar tool such as iPhone's calendar, Outlook,Google Calendar. For any alteration or cancellation of appointmentsbooked through the system, a URL allows the user to be directed to theplatform to do so. Reminders are sent to the user based on his choice ofcontact channel, and provide an opportunity to cancel the bookinginstead of “no-show” at last minute.

Medical vendors who sign up for the system's scheduling services areempowered to use many functionalities that improve customer experiences,employee and customer scheduling efficiencies, and increase revenue bymaximizing capacity utilization and engaging customers.

The business vendor inputs its operating hours, maximum capacity foreach hour (depending on industry, maximum capacity may be furtheritemized by employee or by table, among others), duration for each typeof services. It is anticipated that these inputs are only required to beupdated once in a while. The system allows the company administrator tomanually input a customer booking in the event the customer phones orwalk in person. As such, once the master schedule inputs are completed,the company is able to view its appointment book on a real-time updatedbasis. There are fewer occurrences of writing down the wrong time, nameor phone number of customers.

Business vendors also have the choice of putting a “book now” button(powered by the instant platform) on their company websites. Once acustomer presses on the “book now” button, he is able to schedule anappointment with that business vendor on an interface powered by theinstant. Even if the business vendor chooses not to have its businesslisted on the platform visible to all users, the business vendor'scustomers can still schedule appointment with this vendor by pressing onthe “book now” button.

For businesses who want more control over appointments, they can opt tohave the ability to reject or decline an appointment. After opting forsuch flexibility and if business vendor does not respond to theappointment request in time (specified by the business vendor), theappointment will be deemed as accepted.

The system provides customized services which are locally optimized tosuit an individual user's requirements and yet which globally optimizethe utilization of the system resources supporting such customizedservices for each individual seeking customized services. With thesystem, users will get the simplicity and convenience of schedulingappointments within one platform instead of having to go into multiplewebsites or applications. Once appointments are made, users will alsoeasily integrate the appointment details within the users' existingcalendar tools (such as iCal). In addition to scheduling appointment foran individual user's own purpose, the platform also allows users tocoordinate events with their friends and make the appointment directlyon the system (after the venue, date and time are voted on and chosen onthe system).

The system provides a holistic scheduling platform that allowsbusinesses from all industries to sign-up and customers can scheduleappointments with these businesses or vendors through the system'swebsite as detailed below. Mobile users can access the system though amobile application such as an Android or iPhone application. The mobileapp provides a better user experience than mobile websites are capableof.

Users can also access the system through a vendor website through a“Book Now” widget. The “Book Now” widget is a button displayed on thevendor's web site for a user creates an appointment using the system.When the user clicks the “Book Now” button on the vendor's site, anappointment can be created with a link back to the vendor's website. Oneembodiment uses the Open Graph protocol to specify information about thevendor entity. When the vendor includes Open Graph tags on its Web page,the page becomes equivalent to a system's page. This means when a userclicks the “Book Now” button on the vendor's page, a connection is madebetween the vendor's page and the user. The vendor page will appear inthe “Likes and Interests” section of the user's profile, and the vendorhas the ability to publish updates to the user. There are two “Book Now”button implementations: XFBML and Iframe. The XFBML (also available inHTML5-compliant markup) version is more versatile, but requires use ofthe JavaScript SDK. The XFBML dynamically re-sizes its height accordingto whether there are profile pictures to display, gives the vendor theability (through the Javascript library) to listen for like events sothat the system knows in real time when a user clicks the “Book Now”button, and it always gives the user the ability to add an optionalcomment to the book now function. If users do add a comment, the storypublished back to the vendor is given more prominence.

Vendors can access the system through an administrative console. Inthese verticals, for the business vendors who sign-up with the platform,they have the flexibility and choice to do the following:

-   -   1. Input all or some of the business' operating hours and        schedule availability, so that users can automatically schedule        appointment anytime, anywhere.    -   2. Opt for ability to decline or reject appointments.    -   3. Opt for the business not to be displayed on the list of        business vendors, while that business' customers can still        schedule online appointments automatically through a “book        button” that is supplied by the system for display on the        business' website.

The system performs aggregation of different variables and inputs fordifferent industries in order to solve for the same thing: scheduleavailability. Whilst to the user, the platform gives them the sameconvenience of finding the schedule availability so they can book anyvendors.

For the medical offices, the variable inputs that solve for scheduleavailability or the vendor in this industry aggregates the vendor'sstaffs own individual schedule and service duration. The ratio of staffto customer is generally 1:1. Assume a vendor in this industry has 3staffs who perform services. For timeslot 9-10am, Staff A has beenbooked but Staff B and Staff C have not been booked. Then there exists 2remaining available booking slots for 9 am. For exercise gyms, the keyvariable inputs that solve for schedule availability are defined byequipment. The vendor names the equipment and defines it by seatingcapacity and maximum time limit allowed for that equipment per eachbooking. For gym class activities, the key variable inputs that solvefor schedule availability consists of seat capacity per course, durationof course, frequency of course (per a multitude level of units such asdaily, weekly, biweekly, month and also for each of these, a subset ofoccurrence frequency exists such as occurring 2 days per week or 1 dayper week, for example).

In one embodiment, the specific variables and inputs for exemplaryindustry-flows in calculating total availability (by date or by staff orby earliest availability). The system will deduct the online bookingsmade by users and manual bookings input by vendors to constantly arriveat “remaining schedule availability” real-time. The Variables and Inputsinclude:

-   -   Total number of staff (service providers)    -   Each staffs availability on each day and time    -   Each staffs list of services provided (i.e. each staff is tagged        with all the services she/he can provide)    -   Define and listing of each service    -   Duration of each service

Web user and mobile user can use the system to book appointments on thescheduling platform. In one exemplary appointment booking workflowhandles three possible usage scenario: 1) through the system's web site,2) through a “Book Now” button 4132, or 3) through a mobile application.

In one usage scenario, the user visits the system's web site. The usermay browse or search the interface for a service provider to suit theirneeds. In a mobile usage scenario, the user is directed to search forservice providers from a mobile application. In one usage scenario, theuser clicks on a “Book Now” button at a vendor's web site. The user isimmediately transferred to an interface on the system's web site wherethe user can search for a date and time for a suitable appointment. Oncea desired service is found, the user is presented with times and datesof available appointments. Once the desired time and date are chosen,the user is asked to log in to the system using an account. If the userdoes not yet have an account, he/she will create a user account. If theuser account already exists, the user will simply log in. In anotherembodiment, the user logs in to an account previous to reserving a timeand date. Once logged in, the appointment is scheduled with the user andvendor. In some instances, approval for the appointment is not required.If this is the case, the appointment is automatically saved to thecalendar of the user for later viewing or reminder. A notification emailis also sent to the user. In another instance, approval is required bythe vendor. In this case, the appointment is placed on the approvalqueue of the vendor.

One example use scenario is described next. In one embodiment, a webuser visits a web site and search for a vendor or service provider. Oncethe desired provider is found, the user can then search for a time anddate to reserve an appointment with the vendor. If the user does notalready have a user account, he will be asked to create one. The userthen logs in to schedule the appointment with the vendor. If approval isrequired by the vendor, then the appointment is placed on an approvalqueue. If not, the date and time are saved to a user's calendar, and anemail confirming the appointment is sent to the user. From the vendor'sview point, the provider visits the site and creates an account and thenlogs in. Once signed in, the provider profile can be created orreviewed. In the event of a new business profile, the vendor will alsohave to upload its existing schedule data onto the platform. From thispoint, the vendor can also input manually booked reservations into theplatform and update the schedule database.

In addition to hospital patient and equipment monitoring, the system canbe used in other applications. In an employee access application, thesystem enables an employer to selectively grant access to specific roomsof a facility. Additionally, when an employee is in an area where he isnot normally present, the system can flag a warning to the facilityadministrator. Further, the computer of the employee can be accesslocked so that only the employee with the proper wireless authorizationcan work on a particular computer. All employee accesses, physical aswell as electronic, are tracked for regulatory requirements such asHIPAA requirements so that the administrator knows that only authorizedpersonnel are present. Additionally, the employee can be paged in casehe or she is needed through the voice walkie-talkie over the Zigbeenetwork.

In a vending machine monitoring application, the system can monitorvending machines remotely located to a central monitoring system. Forexample, transmitters can be placed within soda machines to monitor thedepletion of soda machines. When a “sold out” indication is present inthe vending machine, the system can transmit refill/reorder requests toa supplier using the wireless network or the POTS network. Also, thestatus of the vending machine can be monitored (e.g., the temperature ofa ice cream machine or soda machine) to notify a supplier or themaintenance department when maintenance is required.

In a prisoner monitoring embodiment, people who are subject toincarceration need to be monitored. The system can constantly monitorthe prisoners to ensure they are present. A prisoner has a wirelessappliance that is secured or unremovably attached to his person. If thewireless appliance is forcibly removed it immediately transmits anotification to the prisoner monitor. In a Home Confinement Monitoringembodiment, a convict can be required not to leave their home. They aremonitored by the wireless appliance attached to the “home prisoners”which are then monitored by a central monitoring center or station whichcan be a sheriffs office. In the home prisoner monitoring system, thewireless appliance is secured to or unremovably attached to the homeprisoner and if they move outside of the range of the network (i.e., theleave the house), no transmission will be received by the wirelesstransceiver and an alarm is issued by the remote home prisoner monitor.In one embodiment, the alarm can be a phone call or an email message orfax message to the monitoring center or station.

In an Animal Monitoring embodiment, the system can monitor the statusand presence of animals in a stock yard or on a farm by the similarmethodologies discussed above. In this embodiment a plurality of animalscan be monitored for presence as well as condition. For example, thesystem can ensure that animals have not wandered off as well asdetermine conditions such as temperature or heart rate of an animal,this can be accomplished by placing a wireless appliance on the animal.

In a utility monitoring embodiment, a wireless appliance is interfacedwith a utility meter and thereafter transmits the current meter readingat predetermined intervals. Because of the low power requirements ofZigbee and the low duty cycle and low data rate required fortransmitting the information, the battery for powering the Zigbee radiotransmitter can last many months or more.

FIG. 20 shows exemplary architectural environment 3146, usable tocollect sensor data of the user 3148. The environment 3146 includes anexample portable electronic device 3150 configured to include one ormore sensors to collect user habitual information. The portableelectronic device 3150 described herein can include, but not limited to,a smart phone, a wireless computing device, a cellular phone, a personaldigital assistant, a personal navigation device, a personal computer, aportable media player, a personal digital assistance, a laptop, or anyother computing device. The portable electronic device 3150 can beconfigured to collect sensor data via sensors integrated on it anddetermine the status of the user to provide appropriate recommendations.In an embodiment, the sensor described herein can include, but notlimited to, accelerometer, barometer, implantable internal and externalsensors, or any other sensor. In an embodiment, the portable electronicdevice 1350 can be configured to include the sensors or may includeinterfaces with other sensor devices to access the sensor data andreceive user habitual information over a communication network 3152.

The communication network 3152 described herein can include any type ofcommunications network(s), including for example, but not limited to,wire-based networks (e.g., public switched telephone, cable, datanetworks, and so on), wireless networks (e.g., cellular, satellite,Wi-Fi, Bluetooth, radio-frequency, and so on), or a combination thereof.

In an embodiment, the sensors described herein can represent anapplication service that can be operated as a part of any number ofonline service providers, such as an e-health service, a map service, asocial networking site, a search engine, or the like. Further, thesensors can include additional modules or can work in conjunction withmodules such as to perform the operations discussed below. In exampleimplementations, the sensors can be implemented at least in part by astatus application executed by servers, or by a status applicationstored in memory of the portable electronic device 3150.

Further, the environment 3146 can include one or more local/remoteservers 3154 and one or more context providers 3156, which may be storedon a separate server or with the representative set of servers that canbe accessible via the communication network 3152. The one or morelocal/remote servers 3154 and the one or more context providers 3156 canstore information collected and generated by the status application andcan be updated on a predetermined time interval, in real-time, orperiodically.

Typically, the user 3148 carries the portable electronic device 3150 ina pocket or a purse, and the portable electronic device 3150 canaccesses the status application or the sensor application such as tostart collecting the sensor data. However, collecting sensor data aboutindividuals can presents privacy concerns, such as transmitting thesensor data of the individuals over the communication network 3152.Options are available to address privacy concerns. The options are thatan individual user may choose to opt-in to participate or to opt-out tonot participate in tracking or sharing of sensor data. As such, thetracking of the sensor data may require explicit user consent.

In the example, the portable electronic device 3150 can be configured tostarts recording when the sensors on the portable electronic device 3150are activated by a wireless signal (e.g., from the user 1348 using theportable electronic device 3150, from a global system for mobilecommunications (GSM) device, from a Bluetooth® hands free device, from aWi-Fi access point in the building, or from a broadcast radio signal).In an example, the portable electronic device 3150 may record the user3148 habits information (for example, the user 3148 enters a building),and may record the user 3148 walking up the stairs, eating food at arestaurant, running on thread mill, coughing, doing yoga in a silentroom, meditating, or the like. Further, different user habitsinformation recorded of the portable electronic device 1350 is describedin conjunction with FIG. 22.

FIG. 21 shows different modules of the portable electronic device 3150as described in the FIG. 20. The portable electronic device 3150 can beconfigured to include a context module 3158, a controller module 3160, astorage module 3162, a communication module 3164, and a user interfacemodule 3166. In an embodiment, the context module 3158 can be configuredto collect one or more habits information of the user 3148. In anembodiment, the one or more habits described herein can include statichabit information and dynamic habit information. The term “static habitinformation/data” described herein can be the information which may notchange or changes very slowly. Inversely, the term “dynamic habitinformation/data” described herein can be the information which changesmuch more often. In an embodiment, the context module 3158 can be incharge of collecting, being aware users' context information andanalyzing new context information.

In an embodiment, the context module can be configured to include asensor module 3168 and an application module 3170. The sensor module3168 described herein can be configured to collect the user dynamichabits information/data using the sensors of the portable electronicdevice 3150. The dynamic habits data can be collected based on, forexample, indications of non-movements, movements, locations, orenvironmental conditions around the user 3148, along with detectingspeech being spoken in proximity to the user 3148. In an embodiment, thecontext module 3158 can be configured to infer activities based on theextracted features of the collected dynamic habits data. The portableelectronic device 3150 can be configured to use a discriminating powerto identify correlations between features of the collected sensor dataand possible activities. Further, the detailed overview of the dynamichabits information of the user 1348 is described in conjunction withFIG. 22 a.

In an embodiment, the application module 3170 can be configured todetermine the static habits data of the user 3148. The applicationmodule 3170 can be configured to interface with different applicationspresent in the portable electronic device 3150 such as to collect thestatic habits data of the user 3148. In an example implementation, theapplication module 3170 can gather information about the inferredactivities of the user 3148 that occurred and are likely to occur in theuser's daily life. For example, the activities described herein caninclude office working time, meeting time, the user 1348 schedules,office tea time, office lunch time, tasks, user birth day, party dates,appointments, and the like. Further, the detailed overview of thedynamic habits information of the user 1348 is described in conjunctionwith FIG. 22 b.

In an embodiment, the controller module 3160 can be configured topredict preventive health care recommendation based on one or more rulesand the one or more habits collected from the user 1348. In anembodiment, the controller module 3160 can be configured to implementone or more rules for the one or more habits received from the user1348. The one or more rules described herein can include one or morecondition/logic to recommend one or more recommendations to the user1348 based on the collected habits information. In an embodiment, thecontroller module 3160 can be configured to include a learning module3172 and a statistic analyzing module 3174.

In an embodiment, the learning module 3172 can be configured to receiveone or more similar habits information associated with one or moreusers. The learning module 3172 can be configured to interface withdifferent electronic devices such as e-health platform, the one or morecontent providers 3156, the one or more local/remote servers 3154,research centers, health information exchange systems, and the like toreceive information about the users having same/similar habits. In anembodiment, the learning module 3172 can be configured to interact withhospital platform such as to receive solutions, suggestion, advices,diseases, medications, treatments, prescriptions, and the likeassociated with the similar/same habits. In an embodiment, thestatistical analyzing module 3174 can be configured to analyze theinformation received from the learning module 3172 over a span of time.The statistical analyzing module 3174 continuously analyzes the datasuch as to quickly provide recommendations/advice associated with thehabits. In an embodiment, the controller module 3160 can be configuredto predict the preventive health care recommendations to the user 1348based on the stored one or more rules and the habits informationreceived from the user 1348.

In an embodiment, the storage module 3162 can be configured to store theone or more rules and the one or more habits received from the user1348. The storage module 3162 can be configured to include a staticstorage module 3176 and dynamic storage module 3178. In an embodiment,the static storage module 3176 can be configured to store static habitsdata received from the user 1348. In an embodiment, the dynamic storage3178 can be configured to store dynamic habits data of the user 3148.

In an embodiment, the communication module 3164 can be configured tocommunicate with local/remote devices, such as to receive/transferinformation to/from the user 1348. The communication module 3164 can beconfigured to include one or more interfaces such as to communicate withthe portable electronic device 3150. In an embodiment, the userinterface module 3166 can be configured to provide a graphical userinterface (GUI) to the user 1348 such as to access the predictedpreventive health care recommendations to the user 1348.

FIG. 22 a shows exemplary dynamic habits data of the user 1348. Forexample, features collected from the context module 3158 can includeboth the static habits data and the dynamic habits data. An example ofthe dynamic habits (which may changes much more often) collected by theportable electronic device 3150 can include, but are not limited to, theuser 108 is driving a bus, car, van, and so on, the user 108 is ridingin escalator, elevator, and so on, the user 108 is walking on road,park, lawn, and so on, the user 108 is stationary, the user 108 isrunning, the user 108 is dinning, the user 108 is coughing, the user 108surrounding weather data, the user 108 is in meeting, the user 108surrounding environment data, route taken by the user 1348, the user 108direction, the user 108 velocity, the user 108 acceleration, directionof movement of the user 3148, locations of the user 3148, environmentalnoise levels surrounding the user 3148, speech being spoken in proximityto the user 3148, words spoken by the user 1348, the user 1348 issitting in an office or a conference room, the user 108 is sitting in aquiet location or in a location with some background noise, the user 108is speaking or other individuals are speaking, and the like dataassociated with the user 1348.

FIG. 22 b shows exemplary static habits data of the user 1348. Anexample of the static habits static habits (which may not change orchanges very slowly) collected by the portable electronic device 3150can include, but are not limited to, the user 3148 memos data, the user3148 favorite apps data, the user 3148 games data, the user 3148 tasksdata, the user 3148 time table data, the user 3148 activities data, theuser 3148 contacts data, the user 3148 interest data, the user 3148meeting data, the user 3148 dates data, the user 3148 mailing addressdata, the user 3148 schedule data, the user 3148 birthday data, the user3148 favorite food data, the user 3148 likes and dislikes data, and thelike.

FIG. 23 shows an exemplary preventive health care recommendation tables.In an example, as one-level two-state context prediction algorithm (alsoreferred as recommendation prediction algorithm) is described in theFIG.23, depending on the user static and dynamic habits and the one ormore rules stored in the system. In first table 3168, there is user'shabits information and user's information that can be referenced touser's habits table 3170 and which may further referenced to user'sstatic habit table 3172, and user's dynamic habit table 3174. The tablesare configured to store the user habits information received from thecontext module 3158. The algorithm is configured to map the informationstored in theses tables such as to provide recommendations to the user1348. The information stored in the static and dynamic habits tables canbe further referred by a preventive healthcare recommendation table 3176such as to provide effective recommendations. The algorithm isconfigured to apply one or more rules which may select effectiverecommendations for the user 1348 based on the user static and dynamichabits. In an embodiment, the one or more rules described herein can bedefined based on the information received from the one or more contentproviders, electronic medical record of the user 1348, historictreatments provided to different patients having similar (orsubstantially similar or same habits). In an example, some of theoperations performed by the recommendation prediction algorithm areshown below:

Step 1: Get user's current information and go to step 2. The usercurrent information can include user's current static and dynamic habitsinformation. The system uses the context module 3158 to receive thestatic and dynamic habits of the user 1348.

Step 2: Check the user current information with the information storedin the tables such as described in the FIG. 23. If the information isdifferent from the information stored in the tables, then store thecurrent information in the tables and go to step 3. For example, thesystem can be configured to check the values received by the contextmodule 3182 with the values already stored in the system, if the valuesare different or the values is not a null value, then store the currentvalues of the user 1348 and go to step 3.

Step 3: Get users' static information (such as scheduled tasks,timetable, time, etc.). Go to step 4. Step 4: Determine if any of thestatic habit matches with the user dynamic habits and go to step 5. Forexample, if the user 1348 received dynamic habit is eating and thestatic habits indicates the user usual time for lunch is in the officecanteen then the system determines that the user 1348 is eating food intheir office canteen.

Step 5: Search the most suitable recommendation depending on the habitsinformation. If results are found, then send them to the user device andgo to step 6. For example, as shown in the FIG. 23, the preventiverecommendation table 3176 can use the one or more rules such as todetermine the recommendations for the user 1348. In an example,according to the one or more rules, if the user 1348 have the habit-1then provide recommendations 1^(st), 2^(nd), and Nth. If the user hashabit-2 then provides recommendation 2nd. If the user has habits 1^(st)and Nth then provide recommendation 1. Similarly, if the user has habits1 and 3 then provide recommendation Nth.

Step 6: Display the recommendations to the user 1348. Request the user1348 to provide information about their health and go to step 7. Forexample, the system provides the recommendations to the user 1348 andrequest to provide health information after performing the recommendedaction. In an example, the user may perform the recommended action andtext the system about their heath after performing the action. Forexample, the user 1348 may send that he/she is feeling well or feelingsomething else after having a mini meal.

Step 7: If the algorithm determines that the user health is still notgood then perform the steps 4-7.

FIG. 24 shows exemplary operations performed by the system as describedin the FIGS. 20 and 21. In an example, at 3178, the context module 3158can be configured to collect the user habit information. In an example,the context module 3158 can retrieve both the static and dynamic habitsof the user via one or more sensors installed thereon. In an embodiment,at 3180, the context module 3158, in communication with the sensormodule 3168, receive one or more dynamic habits information of the user1348. The one or more dynamic habits information described herein caninclude for example, the user 1348 movements, location, direction,environment, activity, and the like. In an embodiment, at 3182, thecontext module 3158, in communication with the application module 3170,receive one or more static habits of the user 1348. The one or morestatic habits described herein can include for example, the user 1348schedule, the user 1348 appointment dates, the user 1348 dates, the user1348 medical records, the user 1348 historic activities, and the like.

In an embodiment, at 3184, the controller module 3160, in communicationwith the context module 3158, can be configured to provide preventivehealth care recommendations to the user 1348. In an example, thecontroller module 3160 can predict the preventive health carerecommendation based on the one or more habits information received fromthe context module 1358. In an embodiment, the controller module 3160can interact with the learning module 3172 such as to determine therecommendations for the one or more habits information of the user 1348.In an embodiment, the learning module 3172 can be configured to includehistoric information about the treatments, solutions, advice, diseases,and the like information associated with the one or more users havingsimilar (or substantially similar/same) habits. In an example, thelearning module 3172 can interface with doctors system and includesdoctors/physicians suggestion, approvals, advice, recommendations forthe user 1348 based on the one or more habits. In an embodiment, thecontroller module 3160, in communication with the statistics analyzingmodule 3174, can be configured to use the statistical information aboutthe one or more received habits of the user 1348 with the solutions,recommendations, advice, and other information offered to the one ormore users, who have similar (or substantially similar/same) habits. Inan embodiment, the controller module 3160 can be configured to apply oneor more rules and determine the one or more recommendations applicablefor the user 1348 based on the one or more habits of the user 1348. Therules specify the recommendations for the user 1348 based on the userhabits information and based on the historic information of the one ormore users having similar or same habits.

In an embodiment, at 3190, the controller module 1360, in communicationwith the communication module 3164, can be configured to provide the oneor more predicted preventive health care recommendations to the user1348. In an embodiment, the recommendations can be provided to the user1348 in the form of an alert message such as an SMS, an email, a voicemessage, a video message, and the like. In an embodiment, at 3192, thecontroller module 1360, in communication with the graphical userinterface module 3166, can be configured to display the one or morerecommendations on the user portable electronic device. In anembodiment, at 3194, the user interface module 3166 can be configured toallow the user 1348 to perform one or more actions. For example, ifbased on the user habits, the recommendation provided to the user 1348are to consult a doctor as soon as possible then the user 1348 can usesthe user interface module 3166 to book an appointment with the doctor.In another example, if based on the user habits, the recommendationprovided to the user to have a mini meal (including a glass of milk andegg) then the user 1348 can take the mini meal and respond to theportable electronic device 1350 about his/her health information aftertaking the mini meal. In an example, the user 1348 can respond using theuser interface module 1366 or by sending an SMS, a voice message, anemail, a video message, or the like.

FIG. 25 shows a flow chart illustrating a method 3196 for providingpreventive health care recommendations to the user 3148. At 3198, themethod 3196 includes collecting information about one or more habits ofthe user 1348. In an example, the method 3196 uses the context module3158 to collect information about the one or more habits of the user1348. In an example, the context module 3158 can be configured toretrieve both the static and dynamic habits of the user via one or moresensors installed thereon. In an embodiment, the method 1396 uses thesensor module 3168 to receive the one or more dynamic habits of the user1348. The one or more dynamic habits described herein can include forexample, the user 1348 movements, location, direction, environment,activities, and the like. In an embodiment, the method 1396 uses theapplication module to receive one or more static habits of the user1348. The one or more static habits described herein can include forexample, the user 1348 schedule, the user 1348 appointment dates, theuser 1348 dates, the user 1348 medical records, the user 1348 historicactivities, and the like.

In an embodiment, at 3200, the method 3196 includes storing one or morerules for one or more habits of the user 1348. In an embodiment, themethod 1396 uses the storage module 3162 to store the one or more rules,which defines the one or more recommendations applicable for the user1348. Further, the method 1396 enables the storage module 3162 to storethe one or more habits of the user 1348.

In an embodiment, at 3202, the method 3196 includes determining if anyof the one or more rules applies for the one or more habits receivedfrom the user 1348. The method 1396 uses the controller module 3160 todetermine if any of the rules applies for the one or more habitsreceived from the user 1348. In an embodiment, in response todetermining that none of the rules applies for the one or more habitsreceived from the user, the method 1396 includes repeating the steps3198 through 3202.

In an embodiment, in response to determining that the one or more rulesapplies for the one or more habits received from the user 1348, themethod 1396 includes predicting preventive care recommendations based onthe one or more habits and the one or more rules. The method 1396 allowsthe controller module 3160 to predict the preventive health carerecommendation based on the one or more habits information received fromthe context module 1358. In an embodiment, the method 1396 uses thelearning module 3172 such as to determine the recommendations for theone or more habits. In an embodiment, the learning module 3172 includeshistoric information about the treatments, solutions, advice, diseases,and the like information associated with the one or more users havingsimilar (or substantially similar/same) habits such as the habits of theuser 1348. Further, the method 1396 allows the learning module 3172 tointerface with the doctors system and includes doctors/physicianssuggestion, approvals, advice, and recommendations for the user 1348based on the one or more habits. In an embodiment, the method 1396 usesthe statistics analyzing module 3174 use the statistical informationabout the one or more received habits of the user 1348 with thesolutions, recommendations, advice, and other information offered to theone or more user, who may have similar (or substantially similar/same)habits. In an embodiment, the method 1396 includes applying the one ormore rules and determining the one or more recommendations applicationfor the user 1348 (based on the one or more habits received of the user1348). The rules described herein specify the recommendations for theuser 1348 based on the user habits information and based on the historicinformation of the one or more users having similar or same habits.

In an embodiment, at 3206, the method 1396 includes providing thepredicted preventive care recommendations to the user 1348. In anembodiment, the method 1396 uses the controller module 3164 to providethe one or more predicted preventive health care recommendations to theuser 1348. In an embodiment, the recommendations can be provided to theuser 1348 in the form of an alert message such as an SMS, an email, avoice message, a video message, and the like. In an embodiment, themethod 1396 uses the graphical user interface module 3166 to display theone or more recommendations on the user portable electronic device. Forexample, if based on the user habits, the recommendation provided to theuser 1348 are to take some tablets on regular basis then the user 1348uses the user interface module 3166 to order medicines from a chemistshop. In an example, the user 1348 can respond about his/her healthafter taking the medicines. In an example, the user 1348 can respondusing the user interface module 1366 or by sending an SMS, a voicemessage, an email, a video message, or the like.

In an embodiment, different examples of the system and method fordetermining predictive health care recommendation for the user based onpatient/user natural habits and one or more stored rules are described.In an example, analysis and display of the blood glucose level isdescribed. The level is automatically stored in the storage module 3162.The system is configured to constantly monitor user health informationand apply one or more rules such as to provide preventive health carerecommendations to the user. For example, if the blood glucose is abovea threshold (e.g., 250 mg/dl), meal tagging and feel tagging informationor any other inquiries and information can be requested and input by theuser. The user may provide the information to the system. If the inputfrom the user indicates that it is more than two hours since the meal,or if the patient is feeling unwell, then the system execute the one ormore rule and may recommend the patient to undertake ketone testing. Ifthe patient is feeling fine and recently ate, then a message can bedisplayed stating, e.g., “glucose level is slightly high” and theassociated recommendation recommending “exercise or drink water” isprovided. In an example, the system may provide a message displaying“glucose level is slightly low and associated recommendationrecommending drink fruit juice” is provided. In an example, based on therecommendation to undertake ketone testing, the results of ketonetesting and blood glucose, and the patient input can be analyzed by thesystem. The system applies one or more rules and determines appropriaterecommendations are generated advising the user to, for example, but notlimited to, eat, begin exercise, stop exercise, and administermedication.

In another example, the system can be used to track patient preferences,especially those relating to obesity involving recommendations relatedto diet and exercise. The recommendations for eating can be highlyspecific and personalized according to the user favorite food andhabits. For example, eat X calories of carbohydrates selected from “yourfavorite veg. or non-veg. food” (e.g., mashed potatoes and green beans).Eat X calories of proteins selected from your favorite shrimp and eggwhites Similarly recommendations for exercise can include duration andexertion level. These recommendations can be provided to the user byexecuting the one or more rules, which includes access to the userstatic habits (such as the user favorite food) and the user currenthealth conditions such as fatness, weight, and the like. The systemanalyzes the user current health information and executes the one ormore rules to provide appropriate recommendations advising the user toeat kind of food which is rich in protein, calories, and the like andwhich can be easily acquired through the user favorite food items.

In an example, the system can also provide personalized recommendationsbased on the user information. For example, if a user is so obese thathe/she cannot walk, then “start walking” would not be a transmittedrecommendation for the user in response to test results showing that theuser has increased blood glucose. In an example, the user preferencesfor particular food likes and dislikes, along with their personalpreferences for exercise type, exertion level, timings, the usersubjective reactions such as general well-being, lethargy,light-headedness, nausea, severe headache, and the like parameters areconsidered in the one or more rules such as to determine recommendationsfor the user. In an example, the user's reaction to the recommendedcourse of action may be applied in analysis and determination of furtherrecommendations for the user.

The system and method can be used to monitor any type health relateddata of the user such as diabetes level, blood pressure, heart diseases,the user weight, body mass index (BMI), obesity level, hear rate, andthe like and send appropriate reminders and recommendations to takemedications or actions at appropriate time. The system and method canalso be used to interface with other social and medical healthinformation exchange platform such as to take advice from the heal careproviders about the reactions or habits of the user such that effectiverecommendations can be provided to the user.

In an implementation, various user preferences on exercises are enteredand considered and compatible exercises are recommended. For example, ifthe user is going over weight and the user likes running then the systemmay recommend the user to run for at least X minutes a day to maintainhis/her weight. As the user likes vigorous exercise like running, othervigorous exercise like swimming, mountain hiking, wall climbing,triathlon training, and the like can be recommended to the user. Inanother example, if a patient indicates that they like corn because it'ssoft and sweet then the system can recommend other foods with similarattributes such as yams.

In an implementation, the system can constantly monitor the user healthinformation. In an example, the system may uses external or internalsensors, or may enquire from the user to provide user healthinformation. If the system determines that the blood glucose (BG) levelis low then the system may execute the one or more rules and send amessage displaying “your BG is very low” and recommend “eat a simplesugar snack of 15 g and input the health/feeling information after 15minutes”. In an example, the system may recheck (or receive a messagefrom the user) the health information of the user after 15 minutes. Ifthe system determines that the BG is in the target range then the systemmay send a message displaying “Good Work! Your BG is in target range”and may provide recommend “keep consuming X number of calories and Ynumber of proteins every day”. In an example, if the system determinesthat the BG is above a target range then the system may execute the oneor more rules and send a message displaying “Your BG is above the targetrange” and recommend “eat carefully and stop consuming deserts”. In anexample, if the system determines that the BG is very high then thesystem may execute the one or more rules and send a message displaying“Your BG is high” and recommend “check ketones”. In an example, if thesystem determines that the BG is constantly high then the system mayexecute the one or more rules and send a message displaying “your BG isconstantly high” and recommend “Caution! Consult your physicianimmediately”. In an example, the system can be configured toautomatically book appointments with the physician and alerts the userto go and meet the physician at the scheduled time.

In an implementation, the one or more rules may include rules related totrends of the BG. For example, when X number of reading continuouslyshows the BG level less than 60 within 24-48 hours then the system maysend a message displaying “Caution! Consult your physician immediately”.In an example, the system can automatically book appointment with thephysician and alerts the user to go and meet the physician at thescheduled time. In an example, if the Y number of consecutive readingsis greater than 240 then the system then the system may execute the oneor more rules and send a message displaying “Caution! Consult yourphysician immediately”. In an example, if the 3 consecutive readings aregreater than 350 then the system may execute the one or more rules andsend a message displaying “Caution! Consult your physician immediately”.In an example, the system may stop generating further recommendationsupon observing such trends of BG in the user and constantly recommendsthe user to consult the physician.

In an implementation, the one or more rules may include rules related tothe trends of the BG for a specific time period. For example, if the BGis 71-110 for 7 days then a message displaying “7 day results: Greatwork, you have stayed in the target range”. If the BG is 141-240 for 7days then a message displaying “7 days results: you are above the targetrange, eat carefully” and recommend “eat food rich in calories andproteins such as eggs”. Further, the system can be configured to provideuseful treatment tips based on the periodic trends. An example of suchtips may include “Avoid sugars product when the BG is low or high”.Furthermore, the system can be configured to provide periodicrecommendations of medicines and items to the user. For example, thesystem may send a message displaying “Your XXX tablets are about tocomplete” and recommend “press *** to go to the chemist shop and orderthe tablets”.

In an example, the system may constantly monitor the user obesity level.The system is configured to store standard weight and BMI (body massindex) such as to determine the user obesity level. The system maymonitor the user current weight and executes one or more rules such asto determine appropriate recommendations for the user. The system usesthe user current weight and height information and determines the BMIindex of the user. For example, based on the one or more rules, if thesystem determines that the user BMI is greater than or equal to18.5 andless than 24.9 then the system may send a message displaying “Good foryou. Try not getting the weight”. If the system determines that the userBMI is less than 8.5 then the system may send a message displaying “Youare under weight! Try gaining weight” and recommend “Try eating highcalorie and protein foods such as eggs, breads, green leafy vegetables,and the like based on the use favorite food items”. If you want to gainweight then it's important to gain slowly. So the system may providerecommendations suggesting how much amount of calories, proteins, fat,and the like should be consumed by the user.

In an example, if the BMI is greater than 25 and less than 29.9 then thesystem may send a message “you are overweight, try losing some weight”and recommend “Do exercise regularly”. In an example, the system maysuggest the time, type of exercise, and diet the user should follow tomaintain the weight. Further, the system may measures the user waistsize such as to provide appropriate recommendation to the user. Thesystem may consider different parameters such as the user bloodpressure, blood glucose level, the user heart rate, the user cholesterollevel, the user tobacco use, the user diabetes status, the user age, theuser family history ((having a father or brother diagnosed with heartdisease before age 55 or having a mother or sister diagnosed before age65), the user physical activities, and the like to provide furtherexercise related recommendations to the user.

In an example, if the BMI is greater than 30 then the system can send amessage displaying “You are obese and you must lose weight” andrecommend “diet and exercise schedule for the user”. If you want to loseweight then it's important to lose slowly. So the system may providerecommendations suggesting how much amount of calories, proteins, fat,exercise, and the like should be followed by the user. The system mayprovide recommendations such as “lose no more than 1 pound to 2 pounds aweak” or “begin with a goal of losing 10 percent of your currentweight”. In an example, one pound equals 3,500 calories and to lose 1pound a week, you need to eat 500 calories or burn 500 calories. It'sbest to work out some combination of both eating less and being morephysically active. Further, the system may provide daily tips to theuser such as “the healthiest way to lose weight and offers the bestchance of long-term success is to lose it slowly”. The system mayconsider different parameters such as the user blood pressure, bloodglucose level, the user heart rate, the user cholesterol level, the usertobacco use, the user diabetes status, the user age, the user familyhistory (having a father or brother diagnosed with heart disease beforeage 55 or having a mother or sister diagnosed before age 65), the userphysical activities, and the like to provide further exercise relatedrecommendations and tips to the user.

Furthermore, the diet recommendations including the amount of calories,proteins, and the like that needs to be consumed in a day is provided tothe user. The factors that the system may considers while providing thediet related recommendations to the user includes for example, but notlimited to, body size, the user physical activities such as the usercommon activates (e.g., washing and waxing a car for 45-60 minutes,washing windows or floors for 45-60 minutes, gardening for 30-45minutes, wheeling self in wheelchair for 30-40 minutes, pushing astroller 2 miles in 30 minutes, raking leaves for 30 minutes, shovelingsnow for 15 minutes, stair walking for 15 minutes, and so on), usersporting activities (e.g., playing volleyball for 45-60 minutes, playingtouch football for 45 minutes, walking 2 miles in 30 minutes, shootingbaskets for 30 minutes, bicycling 5 miles in 30 minutes, dancing fast(social) for 30 minutes, performing water aerobics for 30 minutes,swimming laps for 20 minutes, playing basketball for 15-20 minutes,jumping rope for 15 minutes, running 2 miles in 15 minutes, and so on),and the like.

Furthermore, examples of the one or more rules considering differentparameters and user habitual information are described. In an example,if the user is diabetic and the user is fasting then the system mayprovide recommendation suggesting “eat X number of calories extra forthe day and do just normal exercise”. If the user diabetes level is highand the user has high blood pressure then the system may providerecommendation suggesting “take insulin immediately and have a glass ofjuice with a mini meal”. If the user is feeling head ache, and the bloodpressure level is high then the system may provide recommendationsuggesting “drink a glass of milk and sleep for an hour”. If the userweight is 150 lbs and then the system may provide recommendationsuggesting “ride a bicycle at 6 mph, you will burn 240 calories”. If theuser weight is 150 lbs and the user can run at 5.5 mph then the systemmay provide recommendation suggesting “Run for 1 hour today so that youwill lose 660 calories”. If the user weight is 150 lbs and the user canwalk at 2 mph then the system may provide recommendation suggesting“Walk 4 km today and to burn 240 calories”. If the user weight is 150lbs and the user can swim at 25 yards/minute then the system may providerecommendation suggesting “swim for at least 1.5 hours to lose 275calories”. If the user weight is150 lbs and the user likes playingtennis single then the system may provide recommendation suggesting“play tennis for 2 hours to burn 400 calories”. If the user BMIindicates user is getting over weight and currently riding in elevatorthen the system may execute the one or more rules to providerecommendation suggesting “You are getting overweight! Decrease theweight by taking stairs instead of the elevator”. If the systemdetermines that the user is preparing weekly menu for the shopping thenthe system executes the one or more rules and provide recommendationssuggesting “You work ten hours a day! So be sure to include healthysnacks and soups to fresh-up yourself during the work”. If the systemdetermines the user is suffering from tooth ache then the system mayuses the rules related to the user brushing times and may providerecommendation suggesting “brush your teeth twice daily to take care ofyour teeth. Brush at least for 3 minutes to clean all your teeth well”.

Furthermore, the system may include one or more rules to provide tips tothe user based on the habits information. If the user is doing exercisethen the system may provide the tip “Keep Going! Your body will developendurance & strength with regular exercise”. The tips to the user areprovided based on the user information such as age, sex, and the like.For example, if the user is doing exercise and the user age is 22, thenthe system may provide a tip displaying “An average adult need 2 hoursand 30 minutes of moderate intensity or 1 hour and 15 minutes ofVigorous intensity aerobic activity every week”. In an example, if theuser is eating snacks or the static habit indicate the user likes eatingsnacks every day then the system may provide a tip displaying “plan yoursnacks for the day only when you need it the most and eat only nutrientdense snack” or “Choose you snack smartly! A good snack is less than 150calories per serving and gives you some protein”.

In an embodiment, an example pseudo code that asks the user for his/herweight and height and then calculates the BMI (body mass index) toprovide associated recommendations to the user is shown below:

-   Display “Please enter your height in inches”-   Store input as height.-   Display “Please enter your weight in pounds”-   Store input as pounds-   Calculate BMI as weight×(height)(height)-   Display “Your BMI is ” and BMI-   Determine Recommendation based on BMI and one or more rules-   Display “You are suggested to” Recommendation(s)

In an example, the pseudo code shows each step that can be included inthe logic. All logic can be done every time. However, logic could beadded to the program so that the program provide differentrecommendations based on the BMI and other habits (such as the user age,the user gender, the user health, the user conditions, the useractivities, the user likes, the user dislikes, the user interests, theuser favorite food, the user feelings, the user location, and the like)associated with the user. For example:

-   Display “Please enter your height in inches”-   Store input as height.-   Display “Please enter your weight in pounds”-   Store input as pounds-   Calculate BMI as weight×(height)(height)-   Display “Your BMI is” and BMI-   If the BMI is greater than or equal to18.5 and less than 24.9

Display “Good for you. Try not getting the weight”

Determine recommendations based on BMI, the user habits, the one or morerules

Display “Keep maintaining the same diet”

-   If the BMI is less than 18.5

Display “You are under weight! Try gaining weight”

Determine recommendations based on BMI, the user habits, the one or morerules

Display “Try eating high calorie and protein foods such as eggs, breads,green leafy vegetables, and the like based on the use favorite fooditems”

Determine number of calories & proteins needs to be consumed by the userbased on the rules

Display “The number(s) of calories & proteins that must be consumed are”calories and proteins

-   If the BMI greater than 25 and less than 29.9

Display “You are overweight, try losing some weight”

Determine recommendations based on BMI, the user habits, the one or morerules

Display “Do exercise regularly”

-   If the BMI is greater than 30

Display “You are obese and you must lose weight”

Determine recommendations based on BMI, the user habits, the one or morerules

Display “Diet and exercise schedule for the user” Diet and Schedule

Determine pounds to be loosed weekly based on the rules

Display “Lose no more than 1 pound to 2 pounds a weak”

-   The indention after the line that begins with “If” indicates that    this logic can only be done when the “if” logic is true.

An example pseudo code for determining the user Blood Glucose (BG) leveland the one or more habits, and executing one or more rules to determineappropriate recommendations for the user is described below:

-   Determine user habits information (including the user BG level)-   If the BG is greater than 250 mg

Display “How long back to have your meals”

Store user input as meals information

If the input from the user indicates that it is more than two hourssince the meal

-   -   Determine recommendations for the user based on one or more        rules    -   Display “Please undertake ketone testing”

If the input from the user indicates the user just had the meal

-   -   Determine recommendations for the user based on one or more        rules    -   Display “Glucose level is slightly high. Please exercise or        drink water”

-   If the blood glucose (BG) level is low

Determine recommendations for the user based on one or more rules

Display “Your BG is very low. Eat a simple sugar snack of 15 g and inputthe health/feeling information after 15 minutes”

Display “Please provide your feeling or health information after takingthe snack”

If the input indicates the user is feeling well and the BG is in thetarget range

Determine recommendations for the user based on one or more rules

Display “Good Work! Your BG is in target range. Please keep consuming Xnumber of calories and Y number of proteins every day”

-   If the BG is above a target range

Determine recommendations for the user based on one or more rules

Display “Your BG is above the target range. Please eat carefully andstop consuming deserts”

-   If the BG is very high

Determine recommendations for the user based on one or more rules

Display “Your BG is high. Check ketones”

An example pseudo code determining trends of the BG and executing one ormore rules to determine appropriate recommendations for the user isdescribed below:

-   Determine the user BG level-   Check trends of the BG-   If 5 reading continuously shows the BG level is less than 60 within    24-48 hours

Determine recommendations for the user based on one or more rules

Display “Caution! Consult your physician immediately”

Book an appointment with doctor

Display “Your appoint is schedule for tomorrow”

Alert the user to go and meet the physician at the scheduled time

-   If 4 reading continuously shows the BG level is greater than 240

Determine recommendations for the user based on one or more rules

Display “Caution! Consult your physician immediately”

Book an appointment with doctor

Display “Your appoint is schedule for Today at 2:00 PM”

Alert the user to go and meet the physician at the scheduled time

-   If 3 reading continuously shows the BG level is greater than 352

Determine recommendations for the user based on one or more rules

Display “Caution! Consult your physician immediately”

Book an appointment with doctor

Display “Your appoint is schedule for Tomorrow at 3:00 PM”

Alert the user to go and meet the physician at the scheduled time

An example pseudo code determining trends of the BG for a specific timeperiod and executing one or more rules to determine appropriaterecommendations for the user is described below:

-   Determine the user BG level-   Check trends of the BG for entire weak-   If the BG is 71-110 within last 7 days

Determine recommendations for the user based on one or more rules

Display “7 day results: Great work, your BG is in the target range.Continue same diet and exercise plan”

-   If the BG is 141-240 for 7 days

Determine recommendations for the user based on one or more rules

Display “7 days results: you are above the target range. Eat food richin calories and proteins such as eggs”

An example pseudo code for determining periodic recommendations isdescribed below:

-   Determine user habits information-   Check stored user information-   Determine recommendations for the user based on one or more rules-   Display “Your XXX tablets are about to complete, Please press *** to    go to the chemist shop and order the tablets”-   If the user presses ***

Order XXX tablets through chemist shop

Display “The XXX tablets are successfully ordered”

An example pseudo code for determining the one or more habits of theuser and executing one or more rules to determine appropriaterecommendations for the user is described below:

-   Determine user habits information-   If the user is diabetic and the user is fasting

Determine recommendations for the user based on one or more rules

Display “Eat X number of calories extra for the day and do just normalexercise”

-   If the user diabetes level is high and the user blood pressure level    is high

Determine recommendations for the user based on one or more rules

Display “Take insulin immediately and have a glass of juice with a minimeal”

-   If the user is feeling head ach and the user blood pressure level is    high

Determine recommendations for the user based on one or more rules

Display “Drink a glass of milk and sleep for an hour”

-   If the user weight is 150 lbs

Determine recommendations for the user based on one or more rules

Display “Ride a bicycle at 6 mph, you will burn 240 calories”

-   If the user weight is 150 lbs and the user can run at 5.5 mph

Determine recommendations for the user based on one or more rules

Display “Run for 1 hour today so that you will lose 660 calories”

-   If the user weight is 150 lbs and the user can walk at 2 mph

Determine recommendations for the user based on one or more rules

Display “Walk 4km today and to burn 240 calories”

-   If the user weight is 150 lbs and the user can swim at 25    yards/minute

Determine recommendations for the user based on one or more rules

Display “swim for at least 1.5 hours to lose 275 calories”

-   If the user weight is 150 lbs and the user likes playing tennis    single

Determine recommendations for the user based on one or more rules

Display “Play tennis for 2 hours to burn 400 calories”

-   If the user BMI 29 and currently riding in elevator

Determine recommendations for the user based on one or more rules

Display “You are getting overweight! Decrease the weight by takingstairs instead of the elevator”

-   If the user is preparing weekly menu for the shopping

Determine recommendations for the user based on one or more rules

Display “You work ten hours a day! So be sure to include healthy snacksand soups to fresh-up yourself during the work”

-   If the user is suffering from tooth ache

Determine recommendations for the user based on one or more rules

Display “Brush your teeth twice daily to take care of your teeth. Brushat least for 3 minutes to clean all your teeth well”

An example pseudo code for determining the one or more habits of theuser and executing one or more rules to determine appropriatetips/education messages for the user is described below:

-   Determine the user habits information-   If the user is doing exercise

Determine tips/education message for the user based on one or more rules

Display “Keep Going! Your body will develop endurance & strength withregular exercise”

The tips to the user are provided based on the user information such asage, sex, and the like. For example:

-   If the user is doing exercise and the user age is 22

Determine tips/education message for the user based on one or more rules

Display “An average adult need 2 hours and 30 minutes of moderateintensity or 1 hour and 15 minutes of Vigorous intensity aerobicactivity every week”

-   If the user is eating snacks or the static habit indicate the user    likes eating snacks every day

Determine tips/education message for the user based on one or more rules

Display “Plan your snacks for the day only when you need it the most andeat only nutrient dense snack” or “Choose you snack smartly! A goodsnack is less than 150 calories per serving and gives you some protein”

“Computer readable media” can be any available media that can beaccessed by client/server devices. By way of example, and notlimitation, computer readable media may comprise computer storage mediaand communication media. Computer storage media includes volatile andnonvolatile, removable and non-removable media implemented in any methodor technology for storage of information such as computer readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, RAM, ROM, EEPROM, flashmemory or other memory technology, CD-ROM, digital versatile disks (DVD)or other optical storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices, or any other mediumwhich can be used to store the desired information and which can beaccessed by client/server devices. Communication media typicallyembodies computer readable instructions, data structures, programmodules or other data in a modulated data signal such as a carrier waveor other transport mechanism and includes any information deliverymedia.

All references including patent applications and publications citedherein are incorporated herein by reference in their entirety and forall purposes to the same extent as if each individual publication orpatent or patent application was specifically and individually indicatedto be incorporated by reference in its entirety for all purposes. Manymodifications and variations of this invention can be made withoutdeparting from its spirit and scope, as will be apparent to thoseskilled in the art. The specific embodiments described herein areoffered by way of example only. The above specification, examples anddata provide a complete description of the manufacture and use of thecomposition of the invention. Since many embodiments of the inventioncan be made without departing from the spirit and scope of theinvention, the invention resides in the claims hereinafter appended.

While this invention has been particularly shown and described withreferences to preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the spirit and scope of theinvention as defined by the appended claims.

What is claimed is:
 1. A system to monitor a client, comprising: amobile appliance; and one or more computer implemented agents eachspecializing in a health condition, each agent communicating withanother computer implemented agent, the client or the treatmentprofessional, wherein at least one computer implemented agent sends aninstruction promoting healthy client behavior.
 2. The system of claim 1,comprising computer readable code to: collect information on the client;and select a treatment template based on treatment plan for people withsimilar characteristics to the client.
 3. The system of claim 1,comprising a camera to automatically collect calorie intake of an itemto be consumed and calorie detection code.
 4. The system of claim 3,comprising code to automatically identify volume and content of theitem.
 5. The system of claim 3, comprising code to automaticallydetermine if the item is part of a recommended nutritional guideline andto send messages suggesting alternatives that replace or supplement theitem to at least meet the recommended nutritional guideline.
 6. Thesystem of claim 1, comprising at least one Micro-Electro-MechanicalSystem (MEMS) device to collect client condition.
 7. The system of claim1, comprising code to model patient movements and convert the patientmovements into energy consumption.
 8. The system of claim 1, comprisingcode to model calorie usage from exercises and adapt client diet inresponse to the exercises.
 9. The system of claim 1, comprising code toaccumulate reward points for the client to encourage healthy activities.10. The system of claim 1, comprising code to compare client progresswith progress for people with a similar health condition to the clientand send normative messages to improve client progress.
 11. A method toprovide automatic messaging to a client on behalf of a healthcaretreatment professional, comprising: setting up one or more computerimplemented agents, each specializing in a health condition, with rulesto respond to a client condition; during run-time, receiving acommunication from the client and in response selecting one or morecomputer implemented agents to respond to the communication; andautomatically generating a response to be rendered on a client mobiledevice to encourage healthy behavior.
 12. The method of claim 11,comprising: selecting a treatment template based on prior treatment forpeople with similar health condition; generating treatment planautomatically for the client.
 13. The method of claim 11, comprisingautomatically collecting calorie intake of an item to be consumed with aprocessor controlled camera and calorie detection code.
 14. The methodof claim 13, comprising automatically identifying volume and content ofthe item.
 15. The method of claim 13, comprising automaticallydetermining if the item is in a recommended nutritional guideline andsending messages suggesting alternatives that replace or supplement theitem to at least meet the nutritional guideline.
 16. The method of claim11, comprising automatically collecting data on treatment plancompliance using at least one Micro-Electro-Mechanical System (MEMS)device.
 17. The method of claim 1, comprising generating a diet planpersonalized to the client and meeting a predetermined dietaryguideline.
 18. The method of claim 17, comprising modeling calorie usagefrom client exercises and adapting a client diet plan in response to theexercises.
 19. The method of claim 11, comprising accumulating rewardpoints for the client to encourage healthy activities.
 20. The method ofclaim 11, comprising comparing client progress to people with similarhealth condition and improving client progress through normativemessaging.